Here, we evaluate the effect of different learning rates, i.e. 0, 1e-5, 1e-4 (default) and le-3. Notebly learning rate=0 is equivalent to No RL. Figure shows the number of unique scaffolds with P(active)>0.6 and QED>0.6 for scaffold discovery task with increasing learning rate. For simplicity, we focus on RL DF(scaffold). It can be seen that as the learning rate increases up to 1e-4, more scaffolds with desirable properties are found, indicating the model is guided more efficiently towards the desired chemical space. Meanwhile, the variance between different runs increases. This may be because with a higher learning rate, each update to the model parameters is larger, directing the model's focus towards a more different region of the chemical space. A too high learning rate i.e. le-3 in this study, results in noisy and unstable update. Figure -12d shows the overlap of three runs for the unique compounds generated by RL DF(scaffold) for compound 1. It shows a tendency of reduced overlap as the learning rate increases, indicating each run tends to explore different parts of the chemical space. Additionally, a larger chemical space is explored with a higher learning rate of up to 1e-4. These factors might contribute to the greater variance in the number of unique scaffold with desirable properties between different runs (Figure and Figure ). The results for the molecular optimization task can be found in Supplementary Figure and Figure where similar results are found.