Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Deep learning algorithms are increasingly used in modeling chemical processes. However, black box predictions without rationales have limited used in practical applications, such as drug design. To this end, we learn to identify molecular substructures -- rationales -- that are associated with the target chemical property (e.g., toxicity). The rationales are learned in an unsupervised fashion, requiring no additional information beyond the end-to-end task. We formulate this problem as a reinforcement learning problem over the molecular graph, parametrized by two convolution networks corresponding to the rationale selection and prediction based on it, where the latter induces the reward function. We evaluate the approach on two benchmark toxicity datasets. We demonstrate that our model sustains high performance under the additional constraint that predictions strictly follow the rationales. Additionally, we validate the extracted rationales through comparison against those described in chemical literature and through synthetic experiments. Recently, deep learning has been successfully applied to the development of predictive models relating chemical structures to physical or biological properties, outperforming existing methods BID8 BID14. However, these gains in accuracy have come at the cost of interpretability. Often, complex neural models operate as black boxes, offering little transparency concerning their inner workings. Interpretability plays a critical role in many areas including cheminformatics. Consider, for example, the problem of toxicity prediction. Over 90% of small molecule drug candidates entering Phase I trials fail due to lack of efficacy or due to adverse side effects. In order to propose a modified compound with improved properties, medicinal chemists must know which regions of the molecule are responsible for toxicity, not only the overall level of toxicity BID1. We call the key molecular substructures relating to the outcome rationales. In traditional cheminformatics approaches such as pharmacophore mapping, obtaining such a rationale behind the prediction is an intrinsic part of the model BID24 BID7 BID12 In this paper, we propose a novel approach to incorporate rationale identification as an integral part of the overall property prediction problem. We assume access to the same training data as in the original prediction task, without requiring annotated rationales. At the first glance, the problem seems solvable using existing tools. For instance, attention-based models offer the means to highlight the importance of individual atoms for the target prediction. However, it is challenging to control how soft selections are exploited by later processing steps towards the prediction. In this sense, the soft weighting can be misleading. In contrast, hard selection confers the guarantee that the excluded atoms are not relied upon for prediction. The hard selection of substructures in a molecule is, however, a hard combinatorial problem. Prior approaches circumvent this challenge by considering a limited set of predefined substructures (typically of 1-6 atoms), like the ones encoded in some molecular fingerprints BID7. Ideally, we would like the model to derive these structures adaptively based on their utility for the target prediction task. We formulate the problem of selecting important regions of the molecule as a reinforcement learning problem. The model is parametrized by a convolutional network over a molecular graph in which the atoms and bonds are the nodes and edges of the graph, respectively. Different from traditional reinforcement learning methods that have a reward function provided by the environment, our model seeks to learn such a reward function alongside the reinforcement learning algorithm. More generally, our model works as a search mechanism for combinatorial sets, which readily expands to applications beyond chemistry or graphs. Our iterative construction of rationales provides several advantages over standard architectures. First, sequential selection enables us to incorporate contextual features associated with past selections, as well as global properties of the whole molecule. Second, we can explicitly enforce desirable rationale properties (e.g., number of substructures) by including appropriate regularization terms in the reward function. We test our model on two toxicity datasets: the Tox21 challenge dataset, which is a series of 12 toxicity tests, and the human ether-a-go-go-related gene (hERG) channel blocking. The reinforcement learning model identifies the structural components of the molecule that are relevant to these toxicity prediction tasks while simultaneously highlighting opportunities for molecular modification at these sites. We show that by only selecting about 40-50% of the atoms in the molecules, we can create models that nearly match the performance of models that use the entire molecule. By comparing selected regions with rationales described in chemical literature, we further validate the rationales extracted by the model. Deep Learning for Computing Chemical Properties One of the major shifts in chemical property prediction is towards the use of deep learning. The existing models fall into one of two classes. The first class of models is based on an expert-constructed molecular representation such as fingerprints that encapsulate substructures thought to be important and a range of molecular properties BID33 BID26. These models are not well suited for extracting rationales because desired structures may not be part of the fingerprint. Moreover, it may be challenging to attribute properties recorded in fingerprints to specific substructures in the molecule. One would have to restrict the feature space of the fingerprint, which can harm the performance of the model. The second class of models move beyond traditional molecular fingerprints, instead learning tasktailored representations. Specifically, they employ convolutional networks to learn a continuous representation of the molecule BID8 BID14. BID10's work takes this a step further, and uses the Weisfeiler-Lehman kernel inspired neural model as a way to generate better local representations. Following this direction, our work is also based on learned molecular representation. Our focus, however, is on augmenting these models with rationales. As articulated in the introduction, the task is challenging due to the number of candidate substructures and the need to attribute properties aggregated in convolutions to individual atoms. Reinforcement Learning on Graphs Our work utilizes a similar framework as the reinforcement learning model over graphs described by BID5. However, their work focuses on solving computational problems where the reward is provided by the environment (i.e., a deterministically computable property of the graph, such as max-cut or the traveling salesman problem). In contrast, in our formulation, the rewards are also learned by the system. Another related work in this area BID18 ) utilizes a policy gradient method to search for substructures starting from random points in the graph. Since their model is designed for large graphs that do not fit into memory, it does not consider the global context of the molecule as a whole. This design is limiting in the context of molecular prediction, where such information is valuable for prediction, and where it is also feasible to compute since the graphs are small. Moving away from the convolutional network approach, their model imposes an artificial ordering of the nodes through a sequence network as the reinforcement learning agent traverses the graph. Both of the above models focus on the prediction task, whereas the emphasis of our work is on interpretability. Learning Rationales The topic of interpretability has recently gained significant attention BID20 BID15. The proposed approaches cover a wide spectrum in terms of the desired rationales and the underlying methods. Work in this area include visualization of activations in the network BID11 BID9, and examination of the most influential data examples to provide interpretability BID16. Attention-based models have also been widely used to extract interpretability (Ba et al., Figure 1: Our model makes sequential selections of atoms (light blue) in the molecule and is specified by two networks, the Q-Network and the P-Network. The former constitutes the reinforcement learning agent that assigns a Q-value to each atom, and the latter takes the atom selections of the Q-Network and trains a classifier to predict based solely on those atoms. This prediction is used as a reward that is fed back to the Q-Network.2014; BID4 BID25 BID31 BID32. Our work is mostly closely related to approaches focused on extractive rationales BID19 BID27. BID19 present a model to extract parts of text as rationale, but their model does not readily generalize to graphs, and the sequential nature of our model can place a meaningful ordinal ranking over the atom selections. Our approach uses reinforcement learning as a method to iteratively select important atoms from the graph. We use a Q-learning approach similar to that of BID5. The state of the system at time t corresponds to the atoms selected thus far. The agent takes an action a t at each time step t, where the action is the selection of an atom that has not already been selected. After the agent takes an action, the state s t is updated to include the newly selected atom. Unlike traditional reinforcement learning algorithms in which the agent receives a reward from the environment, we use a separate model (P-Network) to generate the reward r t to the agent. The P-network learns to predict molecular properties such as toxicity based on the selected atoms, and rewards the agent according to the accuracy of its predictions. The agent itself learns a Q-Network to represent the action-value function Q(s, a) needed to maximize the reward throughout the process. In this case, maximizing the reward is equivalent to selecting atoms that help P-Network make accurate predictions based on the partial selections of atoms. The overall approach is illustrated in Figure 1.In the following sections, we will describe the two networks, the Q-Network that guides the agent's actions, and the P-Network that maps agent's selections to predicted molecular properties. The two networks are trained iteratively, in a feedback loop, so that the Q-network can provide good selections of atoms that, when fed to the P-Network, in good overall predictions of molecular properties. Both networks are parametrized by the graph convolutional network that we describe separately below. In a convolutional network, the model updates the feature representation of individual atoms iteratively based on local contexts. The nature of the operations is parameterized and can be learned to support the end-to-end task. We prefer convolutional networks due to their expressiveness and adaptability as compared to traditional molecular fingerprint methods. Define a molecular graph as G = (A, B), in which A is the set of nodes denoting the atoms, and B is the set of edges denoting the bonds between atom pairs. In each successive layer of the network, an atom's representation is updated to incorporate the currently available information from its neighbors, while the bond features remain unchanged. These updates propagate information across the molecule, and allow the network to generalize beyond standard local substructures. Let h l i be a vector-valued atom representation for atom i at layer l, and let A i be the input feature vector for atom i, and B i,j be the input feature vector for the bond between atoms i and j. In this notation, we use h 0 i = A i. (The input atom features include: one-hot encoding of the atomic number, degree, valence, hybridization, aromaticity, and whether or not the atom is in a ring. Bond features include: one-hot encoding of the bond order, aromaticity, and whether or not the bond is in a ring). This initialization differs slightly for the Q-Network so as to incorporate the current state of selections into the convolutional architecture. The update step for atom feature vectors involves a gated unit that receives separate contributions from the atom itself and its neighbors. Specifically, DISPLAYFORM0 where N (i) is the set of neighbors of atom i, W l's are the specific weight matrices that vary across layers (bias weights omitted for brevity), and σ is the sigmoid activation function, applied coordinate-wise. After N iterations, we arrive at final atom representations h N t. The Q-Network is parametrized by the convolutional network described in section 3.1, in which the size of the atom representation at the final layer is set to 1 so that h N i is scalar and interpreted as the Q-value for selecting atom i next. The initial atom features in the Q-Network include the binary indicator of whether the atom has already been selected. In other words, if we define the state s as a binary vector encoding the atoms that have already been selected, then we use an augmented h DISPLAYFORM0 The convolutional network is rerun with these initializations, using the same parameters, after each selection. Thus, despite the fact that the selections are greedy with respect to the Q-values, the model will choose atoms in a manner that is aware of the global context of the molecule as well as state s representing already selected atoms. The P-Network is a prediction network that takes the partial selection of atoms from the Q-Network, and makes a prediction about the label of the molecule using only those atoms selected by the Q-Network. Like the Q-Network, the P-Network is separately parametrized by the convolutional network. To incorporate the selected atoms, we zero out all the initial atom features that are not in the current selection before running the convolutional model. It is important to note that we do not zero out the updated hidden states of these atoms. This allows interaction between disjoint selections on the molecular graph and preserves information related to graph distances. The reasoning behind this is that there are often several substructures of interest on the molecule and their interactions might prove important. We want to allow the network to learn these interactions by facilitating the propagation of information throughout the whole molecule. The P-Network is geared towards predicting molecular properties rather than atomic properties and therefore requires an additional aggregation of the atom vectors h N i. In our model, these atom vectors are first renormalized via an attention mechanism: DISPLAYFORM0 and turned into a neural (adaptable) fingerprint by summing the ing renormalized feature vectors f = iĥ N i. This fingerprint is then passed through a sigmoid function to generate the class prediction,ŷ = σ(W f * f). The prediction loss is measured through standard cross-entropy loss L P = −y log(ŷ) − (1 − y) log(1 −ŷ) where y is the actual label andŷ is the predicted probability of y = 1. The reward for the Q-network is induced by the P-network and defined as: DISPLAYFORM0 where θ refers to the parameters of the P-network and s t,· is the binary state vector updated from s t−1,· in response to action a t. Because we are interested in selecting the important substructures of a molecule, which consist of at least several atoms, we found it useful to train the Q-network after n selections rather than a single selection. The Q-Network is therefore trained using an n-step Q-learning algorithm, in which the network receives a signal for each sequence of n actions that it makes. The loss function for the Q-network is then: DISPLAYFORM1 Where γ is a decay constant, Q t specifies the Q-value of the current state and action, and Q target t+n is the max Q-value of the state n steps from t induced by a separate but jointly trained target QNetwork. During the training process, we use an -greedy search policy, where the agent will choose a random atom with probability instead of the one with the highest Q-value. Keeping the idea of molecular substructures in mind, we find that it is helpful to search for a random neighbor of a selected atom, than a completely random atom. We also employ action-replay using a target Q network that utilizes soft parameter updates in order to increase training stability BID23.In our model, we place limits on the numbers of atoms to be selected. This number is proportional to the number of atoms in the molecule up to a certain ceiling. We note that fixing the number of atoms chosen by the agent is a limitation of our model, but seems to work well in practice. Specifically, we find that taking 40-50% of the atoms up to a limit of 12-15 atoms for larger molecules works well, although this number varies with the problem. We also impose a lower limit of 5 atoms for smaller molecules, because it becomes impossible to distinguish distinct molecules when too few atoms are chosen. Additionally, we impose regularization constraints on the model, in order to enforce certain properties on the selections. Since we are interested in the model selecting specific substructures from the molecule, we impose a penalty to the model for selecting too many disjoint groups. That is, we define a variable C g t = # {disjoint groups of atoms at step t}. We then modify the reward equation 3 as follows: DISPLAYFORM2 Datasets We evaluate our model on two toxicity datasets. The first dataset that we explore is the Tox21 challenge dataset which contains a series of 12 toxicity tests categorized by nuclear response signals and stress response pathways 1. We parse the data using RDKit BID17, removing duplicate entries and cleaning the molecules by removing salts. Because this dataset has data coming from multiple sources, there are conflicting labels for some molecules, which we remove from the dataset entirely. TAB0 contains the information about each of the 12 datasets, highlighting the small number of positive examples in many of the datasets. The second toxicity dataset that we evaluate our model on is the inhibition of the hERG channel 2. Because this protein channel is well-studied, we explore this dataset to see if we can create a predictive model that can generate rationales that match the information in chemical literature. This dataset, taken from BID22, consists of a training set with 3792 molecules, and a test set with 1090 molecules, with 25% positive labels. Since the data was already cleaned, we do no further preprocessing of this dataset. Evaluation Measures: Predictions For each dataset, we compare our model against the top reported systems BID26; BID22. These approaches utilize extensive feature engineering through molecular fingerprints and other computed molecular descriptors. In addition, they use additional training sources for data augmentation. Specifically, BID26 utilize a data augmentation method called kernel-based structural and pharmacological analoging (KSPA), which uses public databases containing similar toxicity tests. We measure the predictive performance of the convolutional model (Mol-CNN, which utilizes the full molecule) to demonstrate that is comparable to the state-of-the-art . Next, we evaluate the performance of our reinforcement learning method (RL-CNN) that makes predictions on a fraction of atoms in the molecule. We compare these different models using the AUC metric, since the datasets contain an unbalanced set of labels. Evaluation Measures: Rationales Because the RL-CNN model makes predictions using only atoms selected as rationales, its quantitative performance indirectly measures the quality of these rationales. However, we are also interested in directly evaluating their quality relative to rationales described in the chemical literature by domain experts. In the ideal case, we would identify rationales that are characteristic to a single class of examples -either positive or negative. Unfortunately, many known toxic substructures are prevalent in both positively and negatively labeled compounds. In fact, BID26 show that adding features representing structural similarity to 2,500 common toxicophores (toxic substructures) to their model does not improve performance on the Tox21 challenge dataset. This shows that the expert-derived "important" regions are not actually sufficient nor necessary for the prediction task. Rationales extracted for the hERG dataset are directly compared with rationales described in the literature. Multiple studies have shown that the presence of a protonable nitrogen atom and 3 hydrophobic cores has a high affinity for binding to this particular protein BID2 BID30. Usually, this nitrogen is secondary or tertiary so that it is more basic. When protonated, the positive charge exhibits cation-pi interactions with certain residues of the hERG channel, which is the crux of the binding mechanism. We show that our model can identify these basic nitrogen atoms within the dataset. For a baseline comparison, we also evaluate rationales obtained by selecting atoms with the strongest influence on the logistic regression model prediction using Morgan fingerprints of radius 3 and length 2048. Morgan fingerprints are boolean vectors constructed by enumerating atom-centered neighborhoods up to a certain radius, assigning integer values to each neighborhood by hashing a categorical feature vector of that neighborhood, and reducing those integers to vector indeces BID29. The importance of an atom can be approximated by the absolute difference between a prediction made with the full molecular fingerprint and a prediction made when substructures containing that atom are excluded from the fingerprint BID28. We restrict the baseline rationales to select the same number of atoms as in the RL-CNN model. Evaluation Measures: Rationales (Synthetic Experiment) Since we do not find well-defined substructures in literature for the tests offered in the Tox21 dataset, we also construct a synthetic experiment. For this experiment, we select specific substructures and set the labels of all molecules containing those substructures to be positive; all other molecules' labels are left unchanged. We specifically focus on 3 toxic substructures: the aromatic diazo group, polyhalogenation, and the aromatic nitro group, two of which are from BID13's work on toxicophores common to multiple toxicity assays. We demonstrate that our model can capture important substructures if the data provides a clear enough signal. Quantitative Evaluation of Convolutional Representation We first demonstrate that our convolutional model performs competitively compared to neural models using molecular fingerprints. Columns four and five in TAB0 compare our for Mol-CNN with the highest performing model DeepTox BID26, both run in the multitask setting. The DeepTox model performs better than our convolutional model by 2.3% on average across the twelve tasks. This is not surprising, as their method uses substantial amounts of additional training data which is not available to our model. The on the hERG dataset are summarized in Table 2. Our model outperforms the top performing model BID22, which uses molecular fingerprints, on the external test set. Quantitative Evaluation of Reinforcement Learning Algorithm For the Tox21 dataset, we use the multi-task instance to compare the of our base convolutional model (Mol-CNN). How- Table 2: Results of different models on the hERG dataset using AUC as the metric. The first 4 models are baselines from BID22 and use molecular fingerprints as input to random forest (RF), support vector machine (SVM), k nearest neighbors (KNN) and associative neural networks (ASNN). For each of their models, we take the average performance for the same model run with different input features. ever, we turn to the single-task instance to evaluate the performance of our rationale model. This is due to the fact that different toxicity tests warrant different important substructures. Therefore, we run individual models for each of the toxicity tests using the base convolution model as well as the reinforcement learning model. We observe a small decrease in performance, ing in a 0.7% decrease in AUC on average, using around 50% of the atoms in the dataset as seen in TAB0. On the hERG dataset, we selected 45% of the atoms, and also observe that the reinforcement learning algorithm performs similarly to the convolution network as seen in Table 2, with a 3.4% decrease in AUC. We see a smaller decrease in performance for the Tox21 datasets on average, likely because many of the datasets have comparatively few number of positive examples, so predicting on fewer atoms allows the model to generalize better. Evaluation of Rationales using Human Rationales In the absence of ground truth rationales, we turn to a specific structural motif-a tertiary nitrogen atom-that is known to exhibit cation-pi interactions with residues of the hERG protein when copresent with certain hydrophobic side chains in the correct 3-D conformation BID6 BID2. In the dataset we used, these tertiary nitrogen substructure occurs more often in positive examples compared to negative examples (78.4 % vs 44.9 %). This suggests that while this substructure is important in positive examples, it is not sufficient to indicate that a molecule is positive. We observe that our model captures this important substructure frequently, and more often in positive examples than negative Figure 2: Two examples of rationales selected by the reinforcement learning model. The selected atoms are highlighted in large light blue circles. In both cases, we see that the model selects the tertiary nitrogen motif, highlighted in small green circles, which is implicated in many inhibitors of the hERG channel. examples (63.6 % vs 46.1 %). Here, we require the model to have selected the nitrogen and at least two of its three carbon neighbors. Figure 2 shows two example selections made by the model. The similar statistical bias in this prediction demonstrates that the model can provide insights at least consistent with prior domain expertise. In contrast, when the fingerprint baseline approach is used, the baseline model matches this substructure less frequently and with no discriminating power between positive and negative examples (19.1 % vs 23.9 %).Evaluation of Rationales using Synthetic Experiments Here, we evaluate how often our model would capture target substructures if those substructures were a sufficient indicator of our target task, toxicity. TAB2 summarizes the , and shows that our model can reliably identify them. Examples of the generated rationales can be seen in Figure 3.The baseline approach matches fewer instances of the aromatic diazo and polyhalogenation motifs, but does identify more of the aromic nitro groups. Substructures symmetrically centered around a single atom -as in the nitro group -directly correspond to a fingerprint index and are well-described by the baseline model. Correlation between adjacent atoms can cause the RL-CNN model to make an incomplete selection; that is, some of the initial atom features implicitly contain information about its neighbors, which leads to these neighbors appearing less important as rationales. Simplifying the initial atom featurization to decrease this correlation causes the model to successfully select the aromatic nitro group in 17/17 cases. Figure 3: From left to right, example rationales generated for the dataset altered based on the presence of aromatic diazo group, polyhalogenation, and aromatic nitro group. The selected atoms are highlighted in large light blue circles; the predefined toxicophores are highlighted in small green circles. This confirms that while fingerprint models can do well when the relevant features happen to coincide with a fingerprint index, our rationale model is superior when the relevant features are less well-captured by the exact features of the fingerprint. We present a model that treats the problem of selecting rationales from molecules as a reinforcement learning problem. By creating an auxiliary prediction network, we use a learned reward structure to facilitate the selection of atoms in the molecule that are relevant to the prediction task, without significant loss in predictive performance. In this work, we explore the applicability of rationales in the chemistry domain. Through various experiments on the Tox21 and hERG datasets, we demonstrate that our model successfully learns to select important substructures in an unsupervised manner, requiring the same data as an end-to-end prediction task, which is relevant to many applications including drug design and discovery. Molecules are far more complicated to reason about as compared to images or text due to complex chemical theories and a lack of definitive ground truth rationale labels. As deep learning algorithms continue to permeate the chemistry domain, it will be ever more important to consider the interpretability of such models.

We use a reinforcement learning over molecular graphs to generate rationales for interpretable molecular property prediction.

Represent the short summary for finding the corresponding scientific article (long):

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.


Question: Represent the scientific article to find a short TLDR.

Question: In recent years, substantial progress has been made on graph convolutional networks (GCN). In this paper, for the first time, we theoretically analyze the connections between GCN and matrix factorization (MF), and unify GCN as matrix factorization with co-training and unitization. Moreover, under the guidance of this theoretical analysis, we propose an alternative model to GCN named Co-training and Unitized Matrix Factorization (CUMF). The correctness of our analysis is verified by thorough experiments. The experimental show that CUMF achieves similar or superior performances compared to GCN. In addition, CUMF inherits the benefits of MF-based methods to naturally support constructing mini-batches, and is more friendly to distributed computing comparing with GCN. The distributed CUMF on semi-supervised node classification significantly outperforms distributed GCN methods. Thus, CUMF greatly benefits large scale and complex real-world applications. In recent years, works on graph convolutional networks (GCN) have achieved great success in many graph-based tasks, e.g., semi-supervised node classification , link prediction and recommendation systems . GCN defines a graph convolution operation, which generates the embedding of each node by aggregating the representations of its neighbors. Given a graph, GCN performs the graph convolution operation layer by layer to obtain the final node representations, which will be passed to neural networks to support various tasks. To perform GCN on large scale graphs in constrained memory or distributed computing environments, different sampling methods have been proposed, such as neighbor sampling and importance sampling (b). Instead of sampling, Cluster-GCN proposes an approach to convert computation on a huge matrix to computing on a set of small matrices. However, these methods still suffer from performance loss when conducting distributed computing. To take use of various contextual information on edges in a graph, Relational GCN (RGCN) extends neighbor aggregation by using edge types in link prediction. Besides the edge types, Edge-enhanced Graph Neural Networks (EGNNs) takes more contextual features into consideration. However, in general, GCN still has the efficiency problem when facing complex forms of contextual information. Besides GCN, graph embedding methods (; b; a;) are also widely applied. In general, these methods rely on first-order and secondorder proximity to embed very large information networks into low-dimensional vector spaces. The first-order proximity in a graph is the local pairwise proximity between two vertices, and the secondorder proximity between a pair of vertices in a graph is the similarity between their neighborhood structures. As for GCN, previous work shows that the graph convolution operation is actually a special form of Laplacian smoothing. Thus, as the converging of the model, the smoothing process can keep the final representation of a node more and more similar to those of its neighbors. Therefore, GCN is consistent with graph embedding methods in capturing the structural information. According to previous work , graph embedding methods have been successfully unified as matrix factorization (MF). Thus, we believe that there might be some connections between GCN and MF. Meanwhile, comparing with GCN, MF-based methods are extremely flexible and suitable for distributed computing (; ;). MF-based methods are also easy and efficient to be extended to tasks with complex forms of contextual information on graph edges (; ; ; ;). Thus, if we can unify the GCN model as a special form of MF, large scale and complex real-world applications will benefit from this. In this paper, we theoretically reveal the connections between GCN and MF, and unify GCN as matrix factorization with co-training and unitization in section 2. Here, the co-training process means co-training with the classification task of labeled nodes as in , and the unitization indicates conducting vector unitization on node representations. Then, under the guidance of our theoretical analysis, we formally propose an alternative model to GCN named Co-training and Unitized Matrix Factorization (CUMF) in section 3. Extensive experiments are conducted on several real-world graphs, and show co-training and unitization are two essential components of CUMF. Under centralized computing settings, CUMF achieves similar or superior performances comparing with GCN. These observations strongly verify the correctness of our theoretical analysis. Moreover, GCN performs poor on dense graphs, while CUMF has great performances. This is may caused by the over-smoothing of graph convolution on dense graphs, while CUMF can balance the smoothing of neighbours and the classification of labeled nodes through the co-training process. Experiments under distributed computing settings are also conducted, and distributed CUMF significantly outperforms the state-of-the-art distributed GCN method, i.e., cluster-GCN . Thus, CUMF is extremely friendly to large scale real-world graphs. Meanwhile, lots of works have been done to model contextual information in MF-based methods (; ; ; ;), which have shown great effectiveness, efficiency and flexibility. In this section, we plan to theoretically unify GCN as a specific form of matrix factorization. First, we need to start from the analysis of how node representations are learned in GCN. According to the definition in previous work , we can formulate each layer of GCN as where ∼ A = A + IN is the adjacency matrix of the graph G with added self-connections, IN is the identity matrix for N nodes in graph G, ∼ D is a diagonal degree matrix with the representation of each node at layer l, W (l) is a layer-specific trainable weight matrix, and σ (·) denotes an activation function (such as ReLU (·) = max (0, ·)). For a node classification task, we can obtain a classification loss where Y is the ground truth labels for the classification task, H (−1) is the representation of each node at the final layer of GCN. Via optimizing Eq., the cross-entropy error of the node classification task can be minimized, and the GCN model can be learned. As in the implementation in previous work (; Veličković et al., 2017;), there is no activation function on the last layer of GCN, and the final representations can be formulated as In, GCN has been proven to be a special form of Laplacian smoothing. The final representations in Eq. tends to converge to which is an approximate solution of the final representations in GCN. Specifically, for each node i in graph G, the approximate solution of the corresponding final representation is from which we have Then, the representation of node i can be obtained as where di is the degree of node i. According to above analysis, to train an approximate GCN model with one loss, which simultaneously models the structure of graph convolution and the node classification task, we can minimize the following loss function where α a hyper-parameter to control the balance between two losses, and the structure loss lstructure refers to where I denotes the set of all the nodes in graph G, and dis (·, ·) is a distance measurement. Here, we apply the commonly used cosine similarity and obtain which is equivalent to To verify whether cosine similarity is consistent with the convergence of GCN during the training procedure, we conduct empirical experiments and train GCN models on the Cora dataset and the Pubmed dataset. Figure 1 demonstrates the average cosine similarity between nodes in the graph and their neighbors, as well as the convergence curves estimated by accuracy during the training of GCN on the two datasets. It is obvious that, the curves on the same dataset share similar tendency. That is to say, as we train a GCN model, the cosine similarity between nodes in the graph and their neighbors is being optimized. This strongly proves that cosine similarity is consistent with the convergence of the GCN model. Then, to simplify the form of Eq., we conduct vector unitization in the learned representations H (−1), and thus each representation h (−1) i has similar l2-norm. Accordingly, through unitization, Eq. is equivalent to which leads to where Ci denotes all the nodes that node i is connected to, and for simplicity. Moreover, for better optimization, we can incorporate negative log likelihood and minimize the following loss function equivalently to Eq. where λ (·) = sigmoid (·). So far, the structure loss lstructure is equivalent to factorizing all the positive edges in the graph G weighted by Usually, in graph embedding methods (; b; a;), negative edges sampling is used, for better convergence. Thus, we can randomly sample negative edges for each edge in graph G. Following previous work in unifying word embedding and graph embedding as implicit matrix factorization, we can rewrite Eq. as and where k is the number of negative samples for each edge, and PG denotes the distribution that generates negative samples in graph G. For each node j, PG (j) = dj / |G|, where |G| is the number of edges in graph G. Then, we can explicitly express the expectation term as from which we have Via combining Eq. and Eq., we can obtain the local structure loss lstructure for a specific edges (i, j) To optimize the objective, we need to calculate the partial derivative of lstructure with respect to After setting Eq. to zero, we can obtain which has two solutions, e v i v j = −1 and which leads to Accordingly, the GCN model can be unified as the following matrix factorization co-training with the classification loss lclass, where node representations in V are unitized. D is a diagonal degree matrix with D i,i = d i. Moreover, according to the in , the unified matrix of GCN in Eq. is as the same as the unified matrix of LINE with the second order proximity (b), except there are two representation matrices in LINE. More specifically, the matrix factorization in Eq. is as the same as LINE with the first order proximity, which is implicit matrix factorization. Our theoretical analysis on GCN is summarized as: Conclusion 2.1 Given a connected graph G with adjacency matrix A, graph convolutional networks can be unified as implicit matrix factorization with co-training with labeled node classification; unitization of node representations. In this section, based on above analysis, we propose an alternative model to GCN named Co-training and Unitized Matrix Factorization (CUMF). Figure 2 provides an overview of the proposed CUMF model, which will be described in detail below. Let x i ∈ R d denote the feature vector of node i and f 1 denote the first MLP in Figure 2. According to our theoretical analysis, given x i, we conduct vector unitization in f 1 (x i) to obtain v i (the representation of node i) as According to the theoretical analysis in section 2, the structural part in our proposed CUMF model should be implicit matrix factorization with negative sampling. Thus, lstructure can be formulated as Furthermore, as in GCN, the classification loss lclass can be obtained as where I L is the set of labeled nodes, y i is the classification label of node i and f 2 is the second MLP in Figure 2. Combining Eq., Eq. and Eq., we obtain the loss function of the proposed CUMF model. That is to say, the proposed CUMF model needs to co-train the classification loss lclass with the structural loss lstructure. In practice, like many semi-supervised models , when we do co-training, the structural loss lstructure and the classification loss lclass are alternately optimized. To be more specific, we frist pick b batches of positive and negative edges, where we sample k additional negative edges for each positive edge according to Eq.. For each batch, we take a gradient step for lstructure. Then we pick a batch of labeled instances and take a gradient step for lclass. We repeat this process until convergence. In summary, the proposed CUMF model is based entirely on our theoretical analysis in section 2. Comparing with previous graph modeling methods (; ; b; a; ;), the unique features of our proposed CUMF method are mainly illustrated in the unitization of node representations, which is derived exactly from our theoretical analysis. Therefore, under the effective guidance of our theoretical analysis on GCN, the proposed CUMF model is extremely clear, concise and reasonable. 4 RELATED WORK GCN updates node representations with the aggregation of its neighbors. Based on GCN, Graph Attention Network (GAT) (Veličković et al., 2017) proposes to aggregate neighbors with taking the diversity of nodes into account. The GCN model needs the whole graph structure. However, in large scale and complex real-world applications, we have millions of nodes and billions of graph edges. Thus, GCN is both time and space consuming, and is hard to perform in constrained memory or distributed computing. Random sampling and importance sampling (b) propose sampling approaches to reduce the computation of aggregation. Instead of approximating the node representations, variance controlled GCN (a) uses sampled node to estimate the change of node representations in every updating step. Cluster-GCN uses graph partition method to split the whole graph into a set of small sub-graphs, where aggregation happens within each small sub-graph. Thus, Cluster-GCN supports constructing mini-batches and distributed computing. The original GCN only considers node features, while the features on edges are also important in a graph. To take contextual features on edges into account, RGCN uses edge types to identify nodes' transition matrix. It achieves good performances in link prediction and entity classification. EGNN treats edge features as a tensor and proposes the doubly stochastic normalization to avoid the element values of output features too big. Based on GCN and GAT, EGNN proposes EGNN(C) and EGNN(A) to handle edge tensor respectively. • Q1 What are the roles of different components in the CUMF model (co-training & unitization)? • Q2 How does the performance of our CUMF model compare to that of GCN on different datasets? • Q3 How the two hyper-parameters effect the performance of our method? • Q4 Comparing with GCN, is our proposed CUMF model more friendly to distributed computing? We evaluate our proposed method based on seven datasets. The statistic of the datasets are shown in Table 1. Cora, Citeseer and Pubmed are three standard citation network benchmark datasets. BlogCatalog and Flickr are social networks. The posted keywords or tags in BlogCatalog and Flickr networks are used as node features. Therefore, these five datasets all have node features. By removing their node features and preserving only the node itself, we made Cora-F, Citeseer-F, Pubmed-F, BlogCatalog-F and Flickr-F. In addition, USA and Europe are two air-traffic networks , where each node corresponds to an airport and edge indicates the existence of flights between the airports. So these two datasets do not contain node features. In summary, in terms of whether or not node features are included, we test our model on two types of datasets: • Structural Datasets: Cora-F, Citeseer-F, Pubmed-F, BlogCatalog-F, Flickr-F, USA and Europe. • Attributed Datasets: Cora, Citeseer, Pubmed, BlogCatalog and Flickr. Firstly, to verify the roles of co-training and unitization, we include CUMF-C (Our method without co-training), CUMF-U (Our method without unitization) and CUMF-C-U (Our method without cotraining and unitization) in our comparisons. Specifically, minus C in name means that the model training is conducted in two stages. That is to say, we first optimize the structure loss independently to obtain the final representations of nodes. Then we keep these representations fixed and only update the parameters of classification model when optimizing the classification loss. Accordingly, minus U in name means that the node representation will not be unitized. Secondly, we compare against the strong baseline (Planetoid*) and the state-of-the-art approach (GCN) as in . Lastly, to test the performace of distributed CUMF, we also include the baseline (Random-GCN) and the state-of-the-art method (Cluster-GCN) as in in our comparisons. We implement our methods on Tensorflow. For the other methods, we use the original papers' code from their github pages. We use the Adam optimizer for all methods with weight decay as zero. And for all methods, we report the mean accuracy of 10 runs with random weight initializations. For our methods: mini-batch size is 256, the dimension of embedding is 0.1 * the dimension of feature, dropout rate is 0.5, learning rate ∈ [0.001, 0.01]. As mentioned in section 3, there are two other important hyper-parameters: the number of negative edges (k) and the balance between structure loss and classification loss (b). We will analyze the effects of k and b in next section. Results of our experiments are summarized in Table 2, Table 3, Table 4 and Figure 3, which will be described in detail below. Firstly, we mainly examine the roles of different components of our proposed method (co-training & unitization). Specifically, the impact of different components on the capturing of structural information needs to be verified. Thus we conduct experiments on structural datasets. According to Table 2, CUMF for semi-supervised node classification outperforms other versions of CUMF by a significant margin. This fully verifies that unitization and co-training are two essential components of CUMF. Therefore, the correctness of our theoretical analysis is also verified. Meanwhile, to be noted that, CUMF is superior to GCN on all structural datasets. This shows that CUMF is most likely to make better use of structural information than GCN. After verifying that our proposed CUMF model is not inferior to GCN in capturing structural information, we directly compare CUMF, GCN and Planetoid* on attributed datasets. Based on the shown in Table 3, the performance of CUMF is consistent to that of GCN on Cora, Citeseer and Pubmed, and it has significant improvement on BlogCatalog and Flickr. It is important to note that the average node degree of BlogCatalog or Flickr is much higher than that of the former three datasets. That is to say, BlogCatalog and Flickr are denser graphs. Therefore, CUMF is consistent with GCN on sparse graphs and significantly outperforms GCN on dense graphs. Previous work shows that the graph convolution operation is actually a special form of Laplacian smoothing, and starcking many convolution layers may lead to over-smoothing. From this perspective, the reason might be that dense graphs are easier to make GCN becoming over-smoothing than sparse gaphs. As mentioned in section 3, we optimize the structure loss by mini-batch gradient descent. Comparing with the graph convolution operation in GCN, the capturing of structural information in our approach is much more flexible. Thus, our proposed CUMF model is less likely converge to over-smoothing. The following experimental phenomena in stability analysis also support this view. We also formally analyze the effects of two important hyper-parameters (b & k) on model performance. As mentioned in section 3, b controls the balance between classification loss and structure loss, and k is the number of negative samples for each positive edge. We conduct empirical experiments on attributed datasets, and the are shown in Figure 3. The flat lines in Figure 3 (a) demonstrates that k has little effect on the testing performances of our proposed CUMF model. As shown in Figure 3 (b), for sparse datasets, b has little impact on the testing performances of CUMF. For dense datasets, i.e., BlogCatalog and Flickr, the performances of CUMF model decrease with the increasing of b. The larger value of b indicates the CUMF model pays more attention on capturing structural information. This may give a reason of the poor performances of GCN on dense datasets: the graph convolution operation is easily to be extremely over-smoothing for capturing structural information on dense datasets. On the other hand, CUMF is flexible with the sparsity of graphs via adjusting the parameter b, which is relatively stable. In the end, we verify the capacity of CUMF (our proposed method) and GCN on distributed computing in our experiments. Specifically, as the performances of centralized GCN is close to those of centralized CUMF on Cora, Citeseer and Pubmed, we only conduct empirical experiments on these datasets for convenience of comparisons. And we include Random-GCN and Cluster-GCN as baselines. Besides, our distributed experiments are conducted in synchronous mode. Experimental are shown in Table 4. On these datasets, distributed CUMF does not suffer performance loss, but distributed GCN methods greatly do, comparing with in Table 3. The reason is that CUMF is MF-based, and MF-based methods naturally support constructing mini-batches . However, for GCN, constructing mini-batches is equivalent to solving graph partitioning problem , which is actually a quite challenging work . Moreover, it is most likely that only a small portion of each mini-batch is labeled, which may greatly affect the convergence of GCN. To the best of our knowledge, CUMF is the first work that connects GCN to MF. We theoretically unify GCN as co-training and unitized matrix factorization, and a CUMF model is therefore proposed. We conduct thorough and empirical experiments, which strongly verify the correctness of our theoretical analysis. The experimental show that CUMF achieve similar or superior performances compared to GCN. We also observe that GCN performs poor on dense graphs, while CUMF has great performances. This is may caused by the over-smoothing of graph convolution on dense graphs, while CUMF can balance the smoothing of neighbours and the classification of labeled nodes via co-training. Moreover, due to the MF-based architecture, CUMF is extremely flexible and easy to be applied to distributed computing for large scale real-world applications, and significantly outperforms state-of-the-art distributed GCN methods.

Represent the short summary for finding the corresponding scientific article (long)

We unify graph convolutional networks as co-training and unitized matrix factorization.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

To solve the gradient vanishing/exploding problems, we proprose an efficient parametrization of the transition matrix of RNN that loses no expressive power, converges faster and has good generalization.


Question: Represent the text to find a very short summary (1-2 sentences max)!

Question: Molecular graph generation is a fundamental problem for drug discovery and has been attracting growing attention. The problem is challenging since it requires not only generating chemically valid molecular structures but also optimizing their chemical properties in the meantime. Inspired by the recent progress in deep generative models, in this paper we propose a flow-based autoregressive model for graph generation called GraphAF. GraphAF combines the advantages of both autoregressive and flow-based approaches and enjoys: high model flexibility for data density estimation; efficient parallel computation for training; an iterative sampling process, which allows leveraging chemical domain knowledge for valency checking. Experimental show that GraphAF is able to generate 68\% chemically valid molecules even without chemical knowledge rules and 100\% valid molecules with chemical rules. The training process of GraphAF is two times faster than the existing state-of-the-art approach GCPN. After fine-tuning the model for goal-directed property optimization with reinforcement learning, GraphAF achieves state-of-the-art performance on both chemical property optimization and constrained property optimization. Designing novel molecular structures with desired properties is a fundamental problem in a variety of applications such as drug discovery and material science. The problem is very challenging, since the chemical space is discrete by nature, and the entire search space is huge, which is believed to be as large as 10 33 . Machine learning techniques have seen a big opportunity in molecular design thanks to the large amount of data in these domains. Recently, there are increasing efforts in developing machine learning algorithms that can automatically generate chemically valid molecular structures and meanwhile optimize their properties. Specifically, significant progress has been achieved by representing molecular structures as graphs and generating graph structures with deep generative models, e.g., Variational Autoencoders (VAEs) , Generative Adversarial Networks (GANs) and Autoregressive Models. For example, proposed a Junction Tree VAE (JT-VAE) for molecular structure encoding and decoding. studied how to use GANs for molecular graph generation. You et al. (2018a) proposed an approach called Graph Convolutional Policy Network (GCPN), which formulated molecular graph generation as a sequential decision process and dynamically generates the nodes and edges based on the existing graph substructures. They used reinforcement learning to optimize the properties of generated graph structures. Recently, another very related work called MolecularRNN (MRNN) proposed to use an autoregressive model for molecular graph generation. The autoregressive based approaches including both GCPN and MRNN have demonstrated very competitive performance in a variety of tasks on molecular graph generation. Recently, besides the aforementioned three types of generative models, normalizing flows have made significant progress and have been successfully applied to a variety of tasks including density estimation , variational inference (; ;), and image generation . Flow-based approaches define invertible transformations between a latent base distribution (e.g. Gaussian distribution) and real-world high-dimensional data (e.g. images and speech). Such an JT-VAE ------RVAE ------GCPN -----MRNN -----GraphNVP ------GraphAF ----- invertible mapping allows the calculation of the exact data likelihood. Meanwhile, by using multiple layers of non-linear transformation between the hidden space and observation space, flows have a high capacity to model the data density. Moreover, different architectures can be designed to promote fast training or fast sampling depending on the requirement of different applications. Inspired by existing work on autoregressive models and recent progress of deep generative models with normalizing flows, we propose a flow-based autoregressive model called GraphAF for molecular graph generation. GraphAF effectively combines the advantages of autoregressive and flow-based approaches. It has a high model capacity and hence is capable of modeling the density of real-world molecule data. The sampling process of GraphAF is designed as an autoregressive model, which dynamically generates the nodes and edges based on existing sub-graph structures. Similar to existing models such as GCPN and MRNN, such a sequential generation process allows leveraging chemical domain knowledge and valency checking in each generation step, which guarantees the validity of generated molecular structures. Meanwhile, different from GCPN and MRNN as an autoregressive model during training, GraphAF defines a feedforward neural network from molecular graph structures to the base distribution and is therefore able to compute the exact data likelihood in parallel. As a , the training process of GraphAF is very efficient. We conduct extensive experiments on the standard ZINC dataset. Results show that the training of GraphAF is significantly efficient, which is two times faster than the state-of-theart model GCPN. The generated molecules are 100% valid by incorporating the chemical rules during generation. We are also surprised to find that even without using the chemical rules for valency checking during generation, the percentage of valid molecules generated by GraphAF can be still as high as 68%, which is significantly higher than existing state-of-the-art GCPN. This shows that GraphAF indeed has the high model capability to learn the data distribution of molecule structures. We further fine-tune the generation process with reinforcement learning to optimize the chemical properties of generated molecules. Results show that GraphAF significantly outperforms previous state-of-the-art GCPN on both property optimization and constrained property optimization tasks. A variety of deep generative models have been proposed for molecular graph generation recently (; ; ;). The RVAE model used a variational autoencoder for molecule generation, and proposed a novel regularization framework to ensure semantic validity. proposed to represent a molecule as a junction tree of chemical scaffolds and proposed the JT-VAE model for molecule generation. For the VAE-based approaches, the optimization of chemical properties is usually done by searching in the latent space with Bayesian Optimization . used Generative Adversarial Networks for molecule generation. The state-of-the-art models are built on autoregressive based approaches (b;). (b) formulated the problem as a sequential decision process by dynamically adding new nodes and edges based on current sub-graph structures, and the generation policy network is trained by a reinforcement learning framework. proposed an autoregressive model called MolecularRNN to generate new nodes and edges based on the generated nodes and edge sequences. The iterative nature of autoregressive model allows effectively leveraging chemical rules for valency checking during generation and hence the proportion of valid molecules generated by these models is very high. However, due to the sequential generation nature, the training process is usually slow. Our GraphAF approach enjoys the advantage of iterative generation process like autoregressive models (the mapping from latent space to observation space) and meanwhile calculates the exact likelihood corresponding to a feedforward neural network (the mapping from observation space to latent space), which can be implemented efficiently through parallel computation. Two recent work-Graph Normalizing Flows (GNF) and GraphNVP -are also flow-based approaches for graph generation. However, our work is fundamentally different from their work. GNF defines a normalizing flow from a base distribution to the hidden node representations of a pretrained Graph Autoencoders. The generation scheme is done through two separate stages by first generating the node embeddings with the normalizing flow and then generate the graphs based on the generated node embeddings in the first stage. By contrast, in GraphAF, we define an autoregressive flow from a base distribution directly to the molecular graph structures, which can be trained end-to-end. GraphNVP also defines a normalizing flow from a base distribution to the molecular graph structures. However, the generation process of GraphNVP is one-shot, which cannot effectively capture graph structures and also cannot guarantee the validity of generated molecules. In our GraphAF, we formulate the generation process as a sequential decision process and effectively capture the sub-graph structures via graph neural networks, based on which we define a policy function to generate the nodes and edges. The sequential generation process also allows incorporating the chemical rules. As a , the validity of the generated molecules can be guaranteed. We summarize existing approaches in Table 1. A normalizing flow defines a parameterized invertible deterministic transformation from a base distribution E (the latent space, e.g., Gaussian distribution) to real-world observational space Z (e.g. images and speech). Let f: E → Z be an invertible transformation where ∼ p E is the base distribution, then we can compute the density function of real-world data z, i.e., p Z (z), via the change-of-variables formula: Now considering two key processes of normalizing flows as a generative model: Calculating Data Likelihood: given a datapoint z, the exact density p Z (z) can be calculated by inverting the transformation f, = f −1 θ (z); Sampling: z can be sampled from the distribution p Z (z) by first sample ∼ p E and then perform the feedforward transformation z = f θ . To efficiently perform the above mentioned operations, f θ is required to be invertible with an easily computable Jacobian determinant. Autoregressive flows (AF), originally proposed in , is a variant that satisfies these criteria, which holds a triangular Jacobian matrix, and the determinant can be computed linearly. Formally, given z ∈ R D (D is the dimension of observation data), the autoregressive conditional probabilities can be parameterized as Gaussian distributions: where g µ and g α are unconstrained and positive scalar functions of z 1:d−1 respectively to compute the mean and deviation. In practice, these functions can be implemented as neural networks. The affine transformation of AF can be written as: The Jacobian matrix in AF is triangular, since ∂zi ∂ j is non-zero only for j ≤ i. Therefore, the determinant can be efficiently computed through D d=1 α d. Specifically, to perform density estimation, we can apply all individual scalar affine transformations in parallel to compute the base density, each of which depends on previous variables z 1:d−1; to sample z, we can first sample ∈ R D and compute z 1 through the affine transformation, and then each subsequent z d can be computed sequentially based on previously observed Following existing work, we also represent a molecule as a graph G = (A, X), where A is the adjacency tensor and X is the node feature matrix. Assuming there are n nodes in the graph, d and b are the number of different types of nodes and edges respectively, then A ∈ {0, 1} n×n×b and X ∈ {0, 1} n×d. A ijk = 1 if there exists a bond with type k between i th and j th nodes. Graph Convolutional Networks (GCN) (; ; ; ; Schütt et al., 2017) are a family of neural network architectures for learning representations of graphs. In this paper, we use a variant of Relational GCN (R-GCN) to learn the node representations (i.e., atoms) of graphs with categorical edge types. Let k denote the embedding dimension. We compute the node embeddings H l ∈ R n×k at the l th layer of R-GCN by aggregating messages from different edge types: where E i = A [:,:,i] denotes the i th slice of edge-conditioned adjacency tensor,Ẽ i = E i + I, and is a trainable weight matrix for the i th edge type. Agg(·) denotes an aggregation function such as mean pooling or summation. The initial hidden node representation H 0 is set as the original node feature matrix X. After L message passing layers, we use the the final hidden representation H L as the node representations. Meanwhile, the whole graph representations can be defined by aggregating the whole node representations using a readout function , e.g., summation. Similar to existing works like GCPN (a) and MolecularRNN , we formalize the problem of molecular graph generation as a sequential decision process. Let G = (A, X) denote a molecular graph structure. Starting from an empty graph G 1, in each step a new node X i is generated based on the current sub-graph structure G i, i.e., p(X i |G i). Afterwards, the edges between this new node and existing nodes are sequentially generated according to the current graph structure, i.e., p(A ij |G i, X i, A i,1:j−1). This process is repeated until all the nodes and edges are generated. An illustrative example is given in Fig. 1 GraphAF is aimed at defining an invertible transformation from a base distribution (e.g. multivariate Gaussian) to a molecular graph structure G = (A, X). Note that we add one additional type of edge between two nodes, which corresponds to no edge between two nodes, i.e., A ∈ {0, 1} n×n×(b+1). Since both the node type X i and the edge type A ij are discrete, which do not fit into a flow-based model, a standard approach is to use Dequantization technique to convert discrete data into continuous data by adding real-valued noise. We follow this approach to preprocess a discrete graph G = (A, X) into continuous data z = (z A, z X): We present further discussions on dequantization techniques in Appendix A. Formally, we define the conditional distributions for the generation as: where where, where g µ X, g µ A and g α X, g α A are parameterized neural networks for defining the mean and standard deviation of a Gaussian distribution. More specifically, given the current sub-graph structure G i, we use a L-layer of Relational GCN (defined in Section 3.2) to learn the node embeddings H L i ∈ R n×k, and the embedding of entire sub-graphh i ∈ R k, based on which we define the mean and standard deviations of Gaussian distributions to generate the nodes and edges respectively: where sum denotes the sum-pooling operation, and H To generate a new node X i and its edges connected to existing nodes, we just sample random variables i and ij from the base Gaussian distribution and convert it to discrete features. More specifically, z where is the element-wise multiplication. In practice, a real molecular graph is generated by taking the argmax of generated continuous vectors, i.e.,, where v p q denotes a p dimensional one-hot vector with q th dimension equal to 1..., n,n−1 }, where n is the number of atoms in the given molecule, GraphAF defines an invertible mapping between the base Gaussian distribution and the molecule structures z = (z A, z X). According to Eq. 9, the inverse process from z = (z A, z X) to can be easily calculated as: In GraphAF, since f: E → Z is autoregressive, the Jacobian matrix of the inverse process f −1: Z → E is a triangular matrix, and its determinant can be calculated very efficiently. Given a minibatch of training data G, the exact density of each molecule under a given order can be efficiently computed by the change-of-variables formula in Eq. 1. Our objective is to maximize the likelihood of training data. During training, we are able to perform parallel computation by defining a feedforward neural network between the input molecule graph G and the output latent variable by using masking. The mask drops out some connections from inputs to ensure that R-GCN is only connected to the sub-graph G i when inferring the hidden variable of node i, i.e., i, and connected to sub-graph G i, X i, A i,1:j−1 when inferring the hidden variable of edge A ij, i.e., ij. This is similar to the approaches used in MADE and MAF . With the masking technique, GraphAF satisfies the autoregressive property, and at the same time p(G) can be efficiently calculated in just one forward pass by computing all the conditionals in parallel. To further accelerate the training process, the nodes and edges of a training graph are re-ordered according to the breadth-first search (BFS) order, which is widely adopted by existing approaches for graph generation (b;). Due to the nature of BFS, bonds can only be present between nodes within the same or consecutive BFS depths. Therefore, the maximum dependency distance between nodes is bounded by the largest number of nodes in a single BFS depth. In our data sets, any single BFS depth contains no more than 12 nodes, which means we only need to model the edges between current atom and the latest generated 12 atoms. Due to space limitation, we summarize the detailed training algorithm into Appendix B. In chemistry, there exist many chemical rules, which can help to generate valid molecules. Thanks to the sequential generation process, GraphAF can leverage these rules in each generation step. Specifically, we can explicitly apply a valency constraint during sampling to check whether current bonds have exceeded the allowed valency, which has been widely adopted in previous models (a;). Let |A ij | denote the order of the chemical bond A ij. In each edge generation step of A ij, we check the following valency constraint for the i th and j th atoms: If the newly added bond breaks the valency constraint, we just reject the bond A ij, sample a new ij in the latent space and generate another new bond type. The generation process will terminate if one of the following conditions is satisfied: 1) the graph size reaches the max-size n, 2) no bond is generated between the newly generated atom and previous sub-graph. Finally, hydrogens are added to the atoms that have not filled up their valencies. So far, we have introduced how to use GraphAF to model the data density of molecular graph structures and generate valid molecules. Nonetheless, for drug discovery, we also need to optimize the chemical properties of generated molecules. In this part, we introduce how to fine-tune our generation process with reinforcement learning to optimize the properties of generated molecules. State and Policy Network. The state is the current sub-graph, and the initial state is an empty graph. The policy network is the same as the autoregressive model defined in Section 4.1, which includes the process of generating a new atom based on the current sub-graph and generating the edges between the new atom and existing atoms, i.e., p (X i |G i) and p (A ij |G i, X i, A i,1:j−1). The policy network itself defines a distribution p θ of molecular graphs G. If there are no edges between the newly generated atom and current sub-graph, the generation process terminates. For the state transition dynamics, we also incorporate the valency check constraint. Reward design. Similar to GCPN You et al. (2018a), we also incorporate both intermediate and final rewards for training the policy network. A small penalization will be introduced as the intermediate reward if the edge predictions violate the valency check. The final rewards include both the score of targeted-properties of generated molecules such as octanol-water partition coefficient (logP) or drug-likeness (QED) and the chemical validity reward such as penalties for molecules with excessive steric strain and or functional groups that violate ZINC functional group filters . The final reward is distributed to all intermediate steps with a discounting factor to stabilize the training. In practice, we adopt Proximal Policy Optimization (PPO) , an advanced policy gradient algorithm to train GraphAF in the above defined environment. Let G ij be the shorthand notation of sub-graph G i ∪ X i ∪ A i,1:j−1. Formally, in the RL process of training GraphAF, the loss function of PPO is written as: where r i (θ) = p θ (Xi|Gi) p θ old (Xi|Gi) and r ij (θ) = p θ (Aij |Gij) p θ old (Aij |Gij) are ratios of probabilities output by old and new policies, and V (state, action) is the estimated advantage function with a moving average baseline to reduce the variance of estimation. More specifically, we treat generating a node and all its edges with existing nodes as one step and maintain a moving average baseline for each step. The clipped surrogate objective prevents the policy network from being updated to collapse for some extreme rewards. Evaluation Tasks. Following existing works on molecule generation (; a;), we conduct experiments by comparing with the state-of-the-art approaches on three standard tasks. Density Modeling and Generation evaluates the model's capacity to learn the data distribution and generate realistic and diverse molecules. Property Optimization concentrates on generating novel molecules with optimized chemical properties. For this task, we fine-tune our network pretrained from the density modeling task to maximize the desired properties. Constrained Property Optimization is first proposed in , which is aimed at modifying the given molecule to improve desired properties while satisfying a similarity constraint. Data. We use the ZINC250k molecular dataset for training. The dataset contains 250, 000 drug-like molecules with a maximum atom number of 38. It has 9 atom types and 3 edge types. We use the open-source chemical software RDkit to preprocess molecules. All molecules are presented in kekulized form with hydrogen removed. Baselines. We compare GraphAF with the following state-of-the-art approaches for molecule generation. JT-VAE is a VAE-based model which generates molecules by first decoding a tree structure of scaffolds and then assembling them into molecules. JT-VAE has been shown to outperform other previous VAE-based models (; Gómez-;). GCPN is a state-of-the-art approach which combines reinforcement learning and graph representation learning methods to explore the vast chemical space. MolecularRNN (MRNN), another autoregressive model, uses RNN to generate molecules in a sequential manner. We also compare our model with GraphNVP , a recently proposed flow-based model. Results of baselines are taken from original papers unless stated. Implementation Details. GraphAF is implemented in PyTorch . The R-GCN is implemented with 3 layers, and the embedding dimension is set as 128. The max graph size is set as 48 empirically. For density modeling, we train our model for 10 epochs with a batch size of 32 and a learning rate of 0.001. For property optimization, we perform a grid search on the hyperparameters and select the best setting according to the chemical scoring performance. We use Adam to optimize our model. Full training details can be found in Appendix C. Density Modeling and Generation. We evaluate the ability of the proposed method to model real molecules by utilizing the widely-used metrics: Validity is the percentage of valid molecules among all the generated graphs. Uniqueness is the percentage of unique molecules among all the generated molecules. Novelty is the percentage of generated molecules not appearing in training set. Reconstruction is the percentage of the molecules that can be reconstructed from latent vectors. We calculate the above metrics from 10,000 randomly generated molecules. Table 2 shows that GraphAF achieves competitive on all four metrics. As a flow-based model, GraphAF holds perfect reconstruction ability compared with VAE approaches. Our model also achieves a 100% validity rate since we can leverage the valency check during sequential generation. By contrast, the validity rate of another flow-based approach GraphNVP is only 42.60% due to its one-shot sampling process. An interesting is that even without the valency check during generation, GraphAF can still achieve a validity rate as high as 68%, while previous state-of-the-art approach GCPN only achieves 20%. This indicates the strong flexibility of GraphAF to model the data density and capture the domain knowledge from unsupervised training on the large chemical dataset. We also compare the efficiency of different methods on the same computation environment, a machine with 1 Tesla V100 GPU and 32 CPU cores. To achieve the in Table 2, JT-VAE and GCPN take around 24 and 8 hours, respectively, while GraphAF only takes 4 hours. To show that GraphAF is not overfitted to the specific data set ZINC250k, we also conduct experiments on two other molecule datasets, QM9 and MOSES . QM9 contains 134k molecules with up to 9 heavy atoms, and MOSES is much larger and more challenging, which contains 1.9M molecules with up to 30 heavy atoms. Table 3 shows that GraphAF can always generate valid, unique and novel molecules even on the more complicated data set MOSES. Furthermore, though GraphAF is originally designed for molecular graph generation, it is actually very general and can be used to model different types of graphs by simply modifying the node and edge generating functions Edge-MLPs and Node-MLPs (Eq. 8). Following the experimental setup of Graph Normalizing Flows (GNF) , we test GraphAF on two generic graph datasets: Table 4: Comparison between different graph generative models on general graphs with MMD metrics. We follow the evaluation scheme of GNF (COMMUNITY-SMALL, which is a synthetic data set containing 100 2-community graphs, and EGO-SMALL, which is a set of graphs extracted from Citeseer dataset . In practice, we use one-hot indicator vectors as node features for R-GCN. We borrow open source scripts from GraphRNN (b) to generate datasets and evaluate different models. For evaluation, we report the Maximum Mean Discrepancy (MMD) between generated and training graphs using some specific metrics on graphs proposed by You et al. (2018b). The in Table 4 demonstrate that when applied to generic graphs, GraphAF can still consistently yield comparable or better compared with GraphRNN and GNF. We give the visualization of generated generic graphs in Appendix D. Property Optimization. In this task, we aim at generating molecules with desired properties. Specifically, we choose penalized logP and QED as our target property. The former score is logP score penalized by ring size and synthetic accessibility, while the latter one measures the druglikeness of the molecules. Note that both scores are calculated using empirical prediction models and we adopt the script used in (a) to make comparable. To perform this task, we pretrain the GraphAF network for 300 epochs for likelihood modeling, and then apply the RL process described in section 4.4 to fine-tune the network towards desired chemical properties. Detailed reward design and hyper-parameters setting can be found in Appendix C. Following existing works, we report the top-3 scores found by each model. As shown in Table 5, GraphAF outperforms all baselines by a large margin for penalized logP score and achieves comparable for QED. This phenomenon indicates that combined with RL process, GraphAF successfully captures the distribution of desired molecules. Note that we re-evaluate the properties of the top-3 molecules found by MolecularRNN, which turn out to be lower than the reported in the original paper. Figure 2 (a) and 2(b) show the molecules with the highest score discovered by our model. More realistic molecules generated by GraphAF with penalized logP score ranging from 5 to 10 are presented in Figure 6 in Appendix E. One should note that, as defined in Sec 4.4, our RL process is close to the one used in previous work GCPN (a). Therefore, the good property optimization performance is believed to come from the flexibility of flow. Compared with the GAN model used in GCPN, which is known to suffer from the mode collapse problem, flow is flexible at modeling complex distribution and generating diverse data (as shown in Table 2 and Table 3). This allows GraphAF to explore a variety of molecule structures in the RL process for molecule properties optimization. Constrained Property Optimization. The goal of the last task is to modify the given molecule to improve specified property with the constraint that the similarity between the original and modified molecule is above a threshold δ. and You et al. (2018a), we choose to optimize penalized logP for 800 molecules in ZINC250k with the lowest scores and adopt Tanimoto similarity with Morgan fingerprint as the similarity metric. Similar to the property optimization task, we pretrain GraphAF via density modeling and then finetune the model with RL. During generation, we set the initial states as sub-graphs randomly sampled from 800 molecules to be optimized. For evaluation, we report the mean and standard deviation of the highest improvement and the corresponding similarity between the original and modified molecules in Table 6. Experiment show that GraphAF significantly outperforms all previous approaches and almost always succeeds in improving the target property. Figure 2 (c) visualizes two optimization examples, showing that our model is able to improve the penalized logP score by a large margin while maintaining a high similarity between the original and modified molecule. We proposed GraphAF, the first flow-based autoregressive model for generating realistic and diverse molecular graphs. GraphAF is capable to model the complex molecular distribution thanks to the flexibility of normalizing flow, as well as generate novel and 100% valid molecules in empirical experiments. Moreover, the training of GraphAF is very efficient. To optimize the properties of generated molecules, we fine-tuned the generative process with reinforcement learning. Experimental show that GraphAF outperforms all previous state-of-the-art baselines on the standard tasks. In the future, we plan to train our GraphAF model on larger datasets and also extend it to generate other types of graph structures (e.g., social networks). 10: end for 19: In this section, we elaborate the network architecture and the implementation details of three tasks. Network architecture. The network architecture is fixed among all three tasks. More specifically, the R-GCN is implemented with 3 layers and the embedding dimension is set as 128. We use batch normalization before graph pooling to accelerate the convergence and choose sum-pooling as the readout function for graph representations. Both node MLPs and edge MLPs have two fullyconnected layers equipped with tanh non-linearity. Density Modeling and Generation. To achieve the in Table 2, we train GraphAF on ZINC250K with a batch size of 32 on 1 Tesla V100 GPU and 32 CPU cores for 10 epochs. We optimize our model with Adam with a fixed learning rate of 0.001. Property Optimization. For both property optimization and constrained property optimization, we first pretrain a GraphAF network via the density modeling task for 300 epochs, and then finetune the network toward desired molecular distribution through RL process. Following are details about the reward design for property optimization. The reward of each step consists of step-wise validity rewards and the final rewards discounted by a fixed factor γ. The step-wise validity penalty is fixed as -1. The final reward of a molecule m includes both property-targeted reward and chemical validation reward. We adopt the same chemical validation rewards as GCPN. We define propertytargeted reward as follows: γ is set to 0.97 for QED optimization and 0.9 for penalized logP optimization respectively. We fine-tune the pretrained model for 200 iterations with a fixed batch size of 64 using Adam optimizer. We also adopt a linear learning rate warm-up to stabilize the training. We perform the grid search to determine the optimal hyperparameters according to the chemical scoring performance. The search space is summarised in Table 7. Constrained Property Optimization. We first introduce the way we sample sub-graphs from 800 ZINC molecules. Given a molecule, we first randomly sample a BFS order and then drop the last m nodes in BFS order as well as edges induced by these nodes, where m is randomly chosen from {0, 1, 2, 3, 4, 5} each time. Finally, we reconstruct the sub-graph from the remaining nodes in the BFS sequence. Note that the final sub-graph is connected due to the nature of BFS order. For reward design, we set it as the improvement of the target score. We fine-tune the pretrained model for 200 iterations with a batch size of 64. We also use Adam with a learning rate of 0.0001 to optimize the model. Finally, each molecule is optimized for 200 times by the tuned model. We present visualizations of graphs generated by GraphAF as well as the training graphs in Figure 3 and Figure 4. The visualizations demonstrate that GraphAF has strong ability to learn different graph structures from generic graph datasets. We present more molecule samples generated by GraphAF in the following pages. Figure 5 presents 50 molecules randomly sampled from multivariate Gaussian, which justify the ability of our model to generate novel, realistic and unique molecules. From Figure 6 we can see that our model is able to generate molecules with high and diverse penalized logP scores ranging from 5 to 10. For constrained property optimization of penalized logP score, as shown by Figure 7, our model can either reduce the ring size, remove the big ring or grow carbon chains from the original molecule, improving the penalized logP score by a large margin. (a) Graphs from training set (b) Graphs generated by GraphAF Figure 3: Visualizations of training graphs and generated graphs of EGO-SMALL. (a) Graphs from training set (b) Graphs generated by GraphAF

A flow-based autoregressive model for molecular graph generation. Reaching state-of-the-art results on molecule generation and properties optimization.

Represent the TLDR for finding the corresponding scientific article:

We propose a novel way to incorporate conditional image information into the discriminator of GANs using feature fusion that can be used for structured prediction tasks.