REINVENT is a REINFORCE type strategy that explicitly regularizes optimization by adding a prior policy to the loss function. This prior policy is derived by computing the negative log likelihood from a fixed copy of the initially trained RNN (the prior). This regularization ensures that the RNN being optimized (the agent) maintains what was initially learnt by the prior i.e., how to generate valid SMILES corresponding to the training distribution. A combination of this prior policy and scaled reward (scaled by scaling coefficient sigma (σ)) is then used to define an augmented likelihood, as shown in Equation . This augmented likelihood then acts as a target policy for the agent and the loss function is now defined as the difference between the agent policy and target policy, shown in Equation . Note that we have replaced -∑ 𝑙𝑜𝑔𝑃(𝑎 𝑡 |𝑠 𝑡-1 ) 𝑇 𝑡=0 by the equivalent term 𝑙𝑜𝑔𝑃(𝐴).