Curriculum Learning Formulation. The implementation of CL builds on the REINVENT platform described by Blaschke et al. (see Methods). In CL, a complex task is decomposed into simpler constituent tasks to accelerate training and convergence. The goal is to guide the agent to learn tasks with increasing complexity before providing the Production Objective. Agent Learning progresses through the Curriculum Phase to the Production Phase and is controlled by Curriculum Progression Criteria, checking that the agent achieves an adequate score threshold for each objective (Fig. ). In the former, the agent is trained on simpler sequential tasks with gradually increasing complexity. In the latter, the agent Fig. Curriculum learning target scaffold construction. We define a curriculum where the target dihydro-pyrazoloquinazoline with a phenyl substituent scaffold is decomposed into sequential simpler substructures (highlighted) to guide the agent. The score drops momentarily when a successive substructure objective is introduced as it is unlikely that currently sampled compounds will also possess the new substructure, by chance. By using curriculum learning, the agent is able to find the target scaffold within 1750 epochs while standard policy-based RL is unsuccessful in the same number of epochs (Supplementary Fig. ). reaches a state of productivity, whereby the agent samples compounds in favourable areas of chemical space that satisfy the Production Objective (see Methods for a formal definition of the CL strategy). Agent policy update is maintained in the Production Phase to ensure the agent continues to sample favourable compounds from diverse minima.