The random.prior.new pre-trained Prior was used from the REINVENT 2.0 13 repository which was trained on ChEMBL . We note that for the Aripiprazole Similarity experiment, this enables direct comparison to Double Loop RL as the authors also used the same Prior. The hyperparameters used for Experiment 1: Aripiprazole Similarity and Experiment 3: Dopamine Type 2 Receptor (DRD2) are presented in Table . The hyperparameters used for the Practical Molecular Optimization (PMO) benchmark is presented in Table . The hyperparameters provided in the PMO repository for REINVENT and AHC were used. The hyperparameters for BAR were tuned according to Table . We note that the default σ hyperparameter is 1 as stated in the BAR repository. However, we found that the resulting AUC Top-10 was much lower than all σ values in 6. Thus, we performed hyperparameter tuning using much larger σ values according to the values REINVENT was tuned with.