Reinforcement learning During reinforcement learning (RL), the 'desirability' of generated molecules is quantified by the environment based on various properties and used as a reward (subsection 2.2). The generator is optimised using the policy gradient scheme. In order to control the exploration rate, molecules are generated based on the output of two generators: a generator that is updated based on the reward function and saved as the final generator (the 'exploitation network' or 'agent') and a static generator (the 'exploration network' or 'prior'). The fraction of outputs coming from each generator mimics the mutation rate in evolutionary algorithms and is tuneable. Currently, this has been tested with the fine-tuned model as exploitation network and the pretrained model as exploration network. To improve exploration, the exploitation network can use the outputs from two networks, of which one is constantly updated based on the reward function and the other is only updated every 50 epochs by default (as is shown by Figure in the original paper ). This is the default for the RNN-based generator, but due to the higher computation costs of transformerbased generators, we advise against using this periodically updated generator.