How to generate data if algo itself is involved in the process with a feedback loop?
This does sounds like a bad idea, since you are selecting your data beforehand and hence likely to cause sample bias. Have you looked at anomaly detection approaches?