to create a scoring function that gives high score to molecules with a wide range of molecular properties. The exact initial values are reported in Supplementary material Section 2.3. We use this scoring function in REINVENT and collect the high-scoring molecules generated during 300 epochs of training as the first 𝒰 (depending on the run, this results in the order of 1,000-10,000 molecules). The number of epochs was chosen so that in most cases a (local) maximum has been found, observed as flattening of the learning curve. We run the experiment for two rounds (initialization, and two rounds of feedback queries, 𝑅 = 2), and evaluate the performance as the average oracle score of the generated molecules at initialization and at the end of each round.