Secondly, REINVENT uses an experience replay mechanism whereby a buffer of the top scoring 100 ideas suggestions is maintained and a random sample of 10 SMILES are sampled from this buffer in each RL step and added to the current batch of compounds when computing the loss function and weight updates.