Benefit of distributional vocabulary. To investigate the benefit of using a distributional vocabulary, instead of using the pretrained VQ-VAE, we also train our model using a fixed vocabulary of fragments, which consists of 56k unique fragments (the same set used to pretrain the VQ-VAE). Figure compares the performance of the two models. On average, the model with fixed fragments took 122k steps, while the VQ-VAE only took 71k steps to find a set of 5,000 good molecules (72% improvement). We further analyze the benefit of using discrete latents with a VQ-VAE rather than continuous latents with a Gaussian prior VAE in Appendix A.