We will in the following briefly review reinforcement learning strategies for recurrent neural networks in order to embed our methodological changes into context. RL introduces the paradigm of an episodic task where an agent (here the RNN) decides an action (𝐚 𝐭 ∈ 𝐀) (here, the next SMILES symbol) at time step 𝐭 based on interaction with an environment which informs the agent on the current state (𝐬 𝐭 ∈ 𝐒) (here, the SMILES string) and corresponding reward (𝐫 𝐭 ) (here, computed at the end of the episode (𝐑 𝐓 ∈ [𝟎, 𝟏]) by the scoring function) in a Markov Decision Process . Different RL strategies can then be used to describe how to navigate this landscape. These usually fall into one of two categories: value-based strategies focus on estimating the value of an action given a particular a state (or value of being in a state) and selecting an action so as to maximize the final estimated return (𝐺 = ∑ 𝑟 𝑡 𝑇 𝑡 ), while policy-based RL focusses on identifying the best policy (𝝅) for selecting actions without necessarily consulting a value function to estimate the absolute value of that state/action.