In general, we are interested in understanding whether RL could help to generate additional, diverse highscoring compounds. Table shows the evaluation metrics used for the tasks of scaffold discovery and molecular optimization. For scaffold discovery, the focus is to find a novel scaffold which exhibits high chance to be active and good QED score (i.e. P(active)>0.6 and QED>0.6). Additionally, the improved predicted activity and QED over input compounds are examined in a secondary analysis. For molecular optimization, it is favourable to have close analogues to the input molecule with improved predicted probability to be active and improved QED. Here, "scaffold" represents Murcko scaffold from RDKit which removes the side chains and the "generic scaffold" is the Murcko scaffold which converts all atom types to carbon and all bonds to single. [1] The RL process (1000 steps) takes about 30 mins with about 800 MB GPU memory on a single GPU Nvidia Tesla V100.