In this work, we build on the de novo molecular design platform, REINVENT, and introduce a CL implementation that can address complex tasks where policy-based RL has difficulties to identify suitable molecules due to the complexities of the reward function. The use of CL extends REINVENT's applicability to complex reward functions that were previously infeasible with standard policy-based RL.