Compound diversity is exemplified in Fig. which displays 3 out of the top 10 compounds (based on total score) of each REINVENT-DockStream experiment. In general, the top compounds were generated at later epochs and possess diverse scaffolds from sampling numerous local minima, effectively circumventing mode collapse. Agent exploration is demonstrated by both the training plots which display gradual optimization of docking scores (Fig. ) and the top sampled compounds (Fig. ) which feature diverse scaffolds. Furthermore, agent perceived desirable scaffolds can be generated multiple times, as illustrated by GOLD and rDock (Fig. and). Interestingly, the top GOLD and rDock compounds shown in Fig. and 5e share subtle differences and vary only by 1 or 2 atoms. The scores achieved by these compounds were nearly identical and suggests the agent implicitly learns 3D structural information when docking is incorporated into the scoring function. This is further exemplified by the Glide docking experiments performed against cyclooxygenase 2 (COX2) which is a target for antiinflammation (Fig. ). The proposed sulfonamide moiety (also observed ubiquitously in replicate experiments) is present in the approved drugs, Celecoxib and Valdecoxib, although the latter was discontinued due to cardiovascular toxicity. The agent was initialized based on a random sampling of the ChEMBL dataset and therefore did not possess any preconceived structural bias. The generation of sulfonamide containing compounds suggests agent structural awareness capable of exploiting the shape and electrostatics of the binding cavity. Overall, the top compounds (based on total score) across all experiments contained compounds that optimized every component specified in the scoring function.