In experiments 1 and 3, the baseline algorithms include REINVENT , Augmented Hill Climbing (AHC) , Best Agent Reminder (BAR) , and Double Loop RL as all algorithms are formulated using REINVENT's loss function with shared hyperparameters. Thus, the experiments isolate the effect of SMILES augmentation and experience replay on sample efficiency. Moreover, SMILES-based models are strong baselines as demonstrated in the GuacaMol 37 , MOSES , and PMO 11 benchmarks. In the PMO benchmarking experiment, we compare our method's performance to diverse molecular optimization models. The Appendix includes further details on the dataset, hyperparameters, and ablation studies.