For all experiments, we fixed the starting initial states to the same 5 randomly selected molecules sampled from ZINC and fixed the total number of reward function calls to 10,000. We observe that CMA-ES has lower success rate than MOLRL but higher diversity while MOLRL outperforms CMA-ES on success and uniqueness. Additionally, we investigated the effect of the parameter σ for values 0.25, 0.5 and 1.0 on the agent training convergence for 1000 epochs in the VAE-CYC space as shown in Figure . We observe that the agent converges faster with smaller standard deviation values which further reinforces our observation that in this case the exploration is limited, and the agent exploits solutions early on during training. The results are reported in Table . Figure shows the evolution of each one of the 3 components of the reward function (QED score, SA score and presence of aminopyrimidine substructure as a binary parameter) during training for σ = 1.0. We observe that the agent learns to generate molecules with the desired structure while simultaneously improving or maintaining QED and SA. The RL agent can effectively exploit regions of the latent space that satisfy the reward function while satisfying multiple criteria simultaneously. The diversity of the generated molecules demonstrates the ability of the RL agent to explore several solutions in the latent space. Since PPO maintains benefits of trust region policy optimization (TRPO) we can efficiently control the generation process in regions of latent space of interest. We note that we do not need to pre-define chemical modification rules and the agent reduces the complexity of explicit chemical modification to vector modification in the latent space.