1. Non-stationary distribution: As RL proceeds, the underlying generative model is updated, and the type of molecules generated in later epochs are distinct from the preceding steps (Supporting Figure ). We observed that a surrogate model trained on fixed data will consistently lose predictive accuracy on later epochs (Supporting Figure ), limiting the utility of persisting data collected during the run.