Here, we evaluate the effect of σ in Eq. 4 which balances the desirability of a molecule (enforced by the scoring objective) and the likelihood of this molecule generated from the prior. A default value is 120. Figure shows the results for the molecular optimization task. A lower σ generally results in more unique similar molecules (i.e. Tanimoto similarity > 0.7) for all three RL settings as shown in Figure . This is because a lower σ keeps the agent closer to the prior which is trained to generate similar molecules. Meanwhile, a higher σ generally helps generate more unique compounds with improved properties in Figure . Ultimately, there is no clear trend in the number of unique compounds that are both similar and show improved properties when varying σ in Figure . Among the Increasing learning rate (up to 1e-4) tends to explore the desired chemical space more efficiently while introducing higher variance between different runs. Learning rate 1e-3 leads to dramatic decrease in model performance.