The aripiprazole similarity task is from the GuacaMol benchmark and the objective is to successfully sample aripiprazole. This experiment was used to demonstrate the importance of experience replay and compare Augmented Memory to existing policy-based algorithms. As the code for Double Loop RL is not released, we took the values reported in their paper which holds as the method was also built directly on REINVENT , uses the same pre-trained Prior, and hyperparameters. Moreover, in the studies presenting AHC and BAR , experience replay was not used but we provide an implementation and further compare their performance.