The application of deep learning for drug discovery provides potential to accelerate therapeutics development. One fundamental challenge is molecular design, involving the design and prioritization of candidate molecules for experimental validation. Molecular design entails a multi-parameter optimization (MPO) search in chemical space, estimated to be in the range of 10 23 to 10 60 molecules. Recently, deep learning has been applied towards more efficient methods of sampling chemical space such that it is possible to identify promising candidate molecules faster. Deep generative models using policy-based reinforcement learning (RL) , value based RL , learning a molecular latent space , and other methods including tree search and genetic algorithms have been proposed to generate molecules that possess a desired set of properties. In the policy-based RL paradigm, an agent (a generative model) learns a policy (series of actions to take at given states) to generate molecules that maximize a reward which is typically computed based on a pre-defined reward function. Often, physics-based approximations of binding affinity such as molecular docking are included as a component in the reward function in order to design candidate molecules with enhanced predicted potency. Given sufficiently long training time, these models can learn to generate molecules which satisfy the desired MPO objective. However, in cases with complex reward functions where minima are difficult to find, the resulting small gradients elicit minimal change to the agent policy. Consequently, the agent may spend many epochs sampling from areas in chemical space that are far away from the desired objective. The issue is exacerbated when computationally intensive components are included in the reward function, such as molecular docking. Thus, policy-based RL can be infeasible for complex MPO objectives, leading to suboptimal allocation of computational resources and eventually suboptimal molecules identified for synthesis.