This paper is concerned with a molecular optimization framework using variational autoencoders (VAEs). In this paradigm, VAE allows us to convert a molecular graph into from its latent continuous vector, and therefore, the molecular optimization problem can be solved by continuous optimization techniques. One of the longstanding issues in this area is that it is difficult to always generate valid molecules. The very recent work called the junction tree variational autoencoder (JT-VAE) successfully solved this issue by generating a molecule fragment-by-fragment. While it achieves the state-of-the-art performance, it requires several neural networks to be trained, which predict which atoms are used to connect fragments and stereochemistry of each bond. In this paper, we present a molecular hypergraph grammar variational autoencoder (MHG-VAE), which uses a single VAE to address the issue. Our idea is to develop a novel graph grammar for molecular graphs called molecular hypergraph grammar (MHG), which can specify the connections between fragments and the stereochemistry on behalf of neural networks. This capability allows us to address the issue using only a single VAE. We empirically demonstrate the effectiveness of MHG-VAE over existing methods.
@cite_16: 
@cite_3: Generating novel graph structures that optimize given objectives while obeying some given underlying rules is fundamental for chemistry, biology and social science research. This is especially important in the task of molecular graph generation, whose goal is to discover novel molecules with desired properties such as drug-likeness and synthetic accessibility, while obeying physical laws such as chemical valency. However, designing models to find molecules that optimize desired properties while incorporating highly complex and non-differentiable rules remains to be a challenging task. Here we propose Graph Convolutional Policy Network (GCPN), a general graph convolutional network based model for goal-directed graph generation through reinforcement learning. The model is trained to optimize domain-specific rewards and adversarial loss through policy gradient, and acts in an environment that incorporates domain-specific rules. Experimental results show that GCPN can achieve 61 improvement on chemical property optimization over state-of-the-art baselines while resembling known molecules, and achieve 184 improvement on the constrained property optimization task.
@cite_2: We present a framework, which we call Molecule Deep @math -Networks (MolDQN), for molecule optimization by combining domain knowledge of chemistry and state-of-the-art reinforcement learning techniques (prioritized experience replay, double @math -learning, and randomized value functions). We directly define modifications on molecules, thereby ensuring 100 chemical validity. Further, we operate without pre-training on any dataset to avoid possible bias from the choice of that set. As a result, our model outperforms several other state-of-the-art algorithms by having a higher success rate of acquiring molecules with better properties. Inspired by problems faced during medicinal chemistry lead optimization, we extend our model with multi-objective reinforcement learning, which maximizes drug-likeness while maintaining similarity to the original molecule. We further show the path through chemical space to achieve optimization for a molecule to understand how the model works.
There are two complementary approaches to molecular optimization. One is the combination of VAE and Bayesian optimization, originally developed by , and the other is reinforcement learning, where the construction of a molecule is modeled as a Markov decision process @cite_16 @cite_3 @cite_2 . These two approaches cover complementary application areas, and therefore, they are not conflicting with each other. The key difference is the assumption on function evaluation cost. The former assumes that the cost is so high that the number of function evaluations should be kept at a minimum, while the latter assumes that the cost is negligible and a number of trial-and-errors are allowed. Therefore, the former is more favorable when the evaluation requires wet-lab experiments or computationally heavy first-principles calculation, and the latter is more favorable when the evaluation can be carried out by light-weight computer simulation. Since our paper focuses on the former setting, in the following, we will discuss its literature.