Is it possible to use AI to reverse engineer software?
Remarkably, more or less the scenario you describe is not only feasible and has already been demonstrated (detailed explanation and fascinating videos at link).

However, the fidelity of the copy is currently quite limited:


So for now, your copy will be quite low quality.  However, there is a big exception to this rule: if the software you are copying is itself based on machine learning, then you can probably make a high-quality copy quite cheaply and easy, as I and my co-authors explain in this short article.

Interesting question and I'm quite sure that the correct answer will change rapidly over the next few years.