To address the mode collapse, a diversity filter (DF1) was applied to both strategies to penalize exploitation and hence encourage exploration. DF1 penalizes the score of any of the top 20% of de novo molecules that were similar to previously generated molecules, a threshold chosen based on the nature of docking-based virtual screening where only the very top ranked molecules are considered. This stabilized learning and rescued the drop in uniqueness in most cases (Figures and). With DF1, AHC evidenced a σ-averaged 1.45-fold improvement compared to REINVENT (with a maximum optimization of 192% at σ=180 for AHC, compared to 119% at σ=180 for REINVENT). Similar to without the DF1, AHC still required 80-90 fewer RL steps to achieve a mean docking score achieved by REINVENT over 100 steps.