are sampled the score contribution goes to zero and 𝑙𝑜𝑔𝑃 𝕌 (𝐴) ≈ 𝑙𝑜𝑔𝑃 𝑝𝑟𝑖𝑜𝑟 (𝐴). In this situation, as the loss function (Equation ) is a distance, the agent policy will, in-fact, trend back towards the prior policy which may negate useful learnings. This situation of low scoring molecules being present will occur especially either early in the learning process or when a difficult or highly constrained scoring function is used. Therefore, the heavy regularization effect of low scoring molecules significantly contributes to slow learning in these situations. In turn, focussing learning only on the high scoring molecules (𝑅 𝑇 → 1) will improve learning. It is worth noting that, high scoring molecules are still regularized by the prior policy, as shown in Equation , ensuring prior learnings are not 'forgotten'. As the RL strategies REINFORCE and HC are not explicitly regularized (as they are in REINVENT, BAR and AHC), cost terms can be added to the loss function to achieve regularization. This step is important in practice to maintain some similarities to the training distribution but also to not catastrophically forget chemical principles which will result in invalid structures (due to valency errors etc.). To assess the effectiveness of this, we evaluated the addition of the Kullback-Leibler (KL) divergence between the prior and agent scaled by a scaling coefficient λ, as shown in Equation 6 and as implemented in .