Experience Replay. Experience replay is implemented as a buffer that stores a pre-defined maximum number of the highest rewarding SMILES sampled so far (100 in this work). Usually, during each sampling, a subset of the buffer is replayed to the Agent . In our proposed method, all SMILES in the buffer are augmented and using their corresponding reward, the Agent is updated multiple times according to the loss function given in Equation .