Recent fragment-based molecular generative models have shown significant empirical advantages over atom-by-atom generative models in molecular optimization. However, they operate over a fixed set of fragments which limits the generative capabilities of the models. Shifting away from previous frameworks, we learn a distribution of molecular fragments using vector-quantized variational autoencoders (VQ-VAE) (van den Oord et al., 2017). Our method builds molecular graphs through the addition and deletion of molecular fragments from the learned distributional fragment vocabulary, enabling the generative model to span a much larger chemical space than models with a fixed fragment vocabulary. Considering atomic edits as primitive actions, the idea of using fragments can be thought of as options as a temporal abstraction to simplify the search problem.