As computing the expectation is intractable, it is instead approximated by sampling a batch, B, of trajectories, i.e., SMILES strings. The process of SMILES generation is further defined as an episodic task where reward is only given at the terminal state. In particular, the desirability of a SMILES sequence only applies once the full SMILES string has been sampled and it maps to a valid molecule. Thus, all intermediate rewards are 0. Defining the set of actions taken in a batch, A * as the specific token sequences generated at a given epoch yields Equation :