For the evaluation of the optimization performance of MOLRL, we start by adopting a widely used benchmark on constrained optimization introduced by Jin et al. . The task is to improve the penalized LogP (pLogP) value for a set of 800 molecules while maintaining structural similarity to the original molecules. The pLogP value is defined as the octanol-water partition coefficient, which is a measure of molecular hydrophilicity, penalized by two scores corresponding to synthetic accessibility and presence of long cycles, respectively. It should be noted that although this is a popular benchmark in the literature for evaluating molecule optimization approaches, it suffers from a known limitation: an increase in the pLogP value can be achieved by simply increasing the number of carbon atoms (additional information in SI section S4).