where θ represents the model parameters, t i represents the i-th token of T , and l represents the length of T . Accordingly, the negative log likelihood (NLL) is defined as Figure shows the general RL workflow. The agent is initialized using the transformer prior, which generates similar molecules to an input molecule. The reinforcement learning loop is further performed to tune the agent's focus on narrower chemical space of interest defined by a set of user-specified scoring components. Specifically, in each RL step, a batch of molecules (batch size=128) are sampled from the agent given the input molecule, and then evaluated based on the scoring function. The evaluated score is combined with the prior and the agent's negative log likelihood for loss computation. The loss is defined as Eq. 3 following .