where S(T ) ∈ [0, 1] is a scoring function whose value represents the evaluated desirability of molecule sequence T . It is an aggregation function of multiple scoring components. More details of S can be found in . σ > 0 is a scalar coefficient balancing the desirability with prior likelihood of a sequence, and θ prior are the parameters of the prior. The agent is updated to minimize Eq. 3, as demonstrated previously , which encourages increasing the evaluated score while keeping the agent not very far away from the prior which has learnt to produce valid and similar molecules. Note that at the beginning of the training θ = θ prior , θ prior are kept fixed, while θ are updated.