Question: Represent the scientific article to find a short TLDR.

Question: Discovering causal structure among a set of variables is a fundamental problem in many empirical sciences. Traditional score-based casual discovery methods rely on various local heuristics to search for a Directed Acyclic Graph (DAG) according to a predefined score function. While these methods, e.g., greedy equivalence search, may have attractive with infinite samples and certain model assumptions, they are less satisfactory in practice due to finite data and possible violation of assumptions. Motivated by recent advances in neural combinatorial optimization, we propose to use Reinforcement Learning (RL) to search for the DAG with the best scoring. Our encoder-decoder model takes observable data as input and generates graph adjacency matrices that are used to compute rewards. The reward incorporates both the predefined score function and two penalty terms for enforcing acyclicity. In contrast with typical RL applications where the goal is to learn a policy, we use RL as a search strategy and our final output would be the graph, among all graphs generated during training, that achieves the best reward. We conduct experiments on both synthetic and real datasets, and show that the proposed approach not only has an improved search ability but also allows for a flexible score function under the acyclicity constraint. Discovering and understanding causal mechanisms underlying natural phenomena are important to many disciplines of sciences. An effective approach is to conduct controlled randomized experiments, which however is expensive or even impossible in certain fields such as social sciences and bioinformatics . Causal discovery methods that infer causal relationships from passively observable data are hence attractive and have been an important research topic in the past decades (; ;). A major class of such causal discovery methods are score-based, which assign a score S(G), typically computed with the observed data, to each directed graph G and then search over the space of all Directed Acyclic Graphs (DAGs) for the best scoring: min G S(G), subject to G ∈ DAGs. While there have been well-defined score functions such as the Bayesian Information Criterion (BIC) or Minimum Description Length (MDL) score and the Bayesian Gaussian equivalent (BGe) score for linear-Gaussian models, Problem is generally NP-hard to solve , largely due to the combinatorial nature of its acyclicity constraint with the number of DAGs increasing superexponentially in the number of graph nodes. To tackle this problem, most existing approaches rely on local heuristics to enforce the acyclicity. For example, Greedy Equivalence Search (GES) enforces acyclicity one edge at a time, explicitly checking for the acyclicity constraint when an edge is added. GES is known to find the global minimizer with infinite samples under suitable assumptions , but this is not guaranteed in the finite sample regime. There are also hybrid methods that use constraint-based approaches to reduce the search space before applying score-based methods, e.g., the max-min hill climbing method . However, this methodology lacks a principled way of choosing a problem-specific combination of score functions and search strategies. introduced a smooth characterization for the acyclicity constraint, and Problem can be formulated as a continuous optimization problem w.r.t. the weighted graph adjacency matrix by picking a proper loss function, e.g., the least squares loss. Subsequent works and have also adopted the evidence lower bound and the negative log-likelihood as loss functions, respectively, and used Neural Networks (NNs) to model the causal relationships. Note that the loss functions in these methods must be carefully chosen in order to apply continuous optimization methods. Unfortunately, many effective score functions, e.g., the generalized score function proposed by and the independence based score function given by, either cannot be represented in closed forms or have very complicated equivalent loss functions, and thus cannot be easily combined with this approach. We propose to use Reinforcement Learning (RL) to search for the DAG with the best score according to a predefined score function, as outlined in Figure 1. The insight is that an RL agent with stochastic policy can determine automatically where to search given the uncertainty information of the learned policy, which gets updated promptly by the stream of reward signals. To apply RL to causal discovery, we use an encoder-decoder NN model to generate directed graphs from the observed data, which are then used to compute rewards consisting of the predefined score function as well as two penalty terms to enforce acyclicity. We resort to policy gradient and stochastic optimization methods to train the weights of the NNs, and our output is the graph that achieves the best reward, among all graphs generated in the training process. Experiments on both synthetic and real datasets show that our approach has a much improved search ability without sacrificing any flexibility in choosing score functions. In particular, the proposed approach using BIC as score function outperforms GES with the same score function on linear non-Gaussian acyclic model (LiNGAM) and linear-Gaussian datasets, and also outperforms recent gradient based methods when the causal relationships are nonlinear. Constraint-based causal discovery methods first use conditional independence tests to find causal skeleton and then determine the orientations of the edges up to the Markov equivalence class, which usually contains DAGs that can be structurally diverse and may still have many unoriented edges. Examples include; that use kernel-based conditional independence criteria and the well-known PC algorithm . This class of methods involve a multiple testing problem where the tests are usually conducted independently. The testing may have conflicts and handling them is not easy, though there are certain works, e.g., , attempting to tackle this problem. These methods are also not robust as small errors in building the graph skeleton can in large errors in the inferred Markov equivalence class. Another class of causal discovery methods are based on properly defined functional causal models. Unlike constraint-based methods that assume faithfulness and identify only the Markov equivalence class, these methods are able to distinguish between different DAGs in the same equivalence class, thanks to the additional assumptions on data distribution and/or functional classes. Examples include LiNGAM (; 2011), the nonlinear additive noise model (;, and the post-nonlinear causal model (Zhang and Hyvärinen, 2009).; , other recent NN based approaches to causal discovery include that proposes causal generative NNs to functional causal modeling with a prior knowledge of initial skeleton of the causal graph and that learns causal generative models in an adversarial way but does not guarantee acyclicity. Recent advances in sequence-to-sequence learning have motivated the use of NNs for optimization in various domains (; ;). A particular example is the traveling salesman problem (TSP) which was revisited in the work of pointer networks . Authors proposed a recurrent NN with nonparametric softmaxes trained in a supervised manner to predict the sequence of visited cities. further proposed to use the RL paradigm to tackle the combinatorial problems due to their relatively simple reward mechanisms. It was shown that an RL agent can have a better generalization even when the optimal solutions are used as labeled data in the previous supervised approach. There are many other successful RL applications in recent years, e.g., AlphaGo , where the goal is to learn a policy for a given task. As an exception, applied RL to neural architecture search. While we use a similar idea as the RL paradigm can naturally include the search task, our work is different in the actor and reward designs: our actor is an encoder-decoder model that generates graph adjacency matrices (cf. Section 4) and the reward is tailored for causal discovery by incorporating both a score function and the acyclicity constraint (cf. Section 5.1). We assume the following model for data generating procedure, as in;. Each variable x i is associated with a node i in a d-node DAG G, and the observed value of x i is obtained as a function of its parents in the graph plus an independent additive noise n i, i.e., where x pa(i) denotes the set of variables x j so that there is an edge from x j to x i in the graph, and the noises n i are assumed to be jointly independent. We also assume causal minimality, which in this case reduces to that each function f i is not a constant in any of its arguments. Without further assumption on the forms of functions and/or noises, the above model can be identified only up to Markov equivalence class under the usual Markov and faithful assumptions (; ; in our experiments we will consider synthetic datasets that are generated from fully identifiable models so that it is practically meaningful to evaluate the estimated graph w.r.t. the true DAG. If all the functions f i are linear and the noises n i are Gaussian distributed, the above model yields the class of standard linear-Gaussian model that has been studied in ; ; ; . When the functions are linear but the noises are non-Gaussian, one can obtain the LiNGAM described in Shimizu et al. (2006; 2011) and the true DAG can be uniquely identified under favorable conditions. In this paper, we consider that all the variables x i are scalars; extending to more complex cases is straightforward with a properly defined score function. The observed data X, consisting of a number of vectors, are then sampled independently according to the above model on an unknown DAG, with fixed functions f i and fixed distributions for n i. The objective of causal discovery is to use the observed data X, which gives the empirical version of the joint distribution of x, to infer the underlying causal DAG G. Given a dataset X = {x k} m k=1 where x k denotes the k-th observed sample, we want to infer the causal graph that best describes the data generating procedure. We would like to use NNs to infer the causal graph from the observed data; specifically, we aim to design an NN based graph generator whose input is the observed data and the output is a graph adjacency matrix. A naive choice would be using feed-forward NNs to output d 2 scalars and then reshape them to an adjacency matrix in R d×d. However, this structure failed to produce promising , possibly because the feed-forward NNs could not provide sufficient interactions amongst variables to capture the causal relations. Motivated by recent advances in neural combinatorial optimization, particularly the pointer networks , we draw n random samples (with replacement) {x l} from X and reshape them as s: n is the vector concatenating all the i-th entries of the vectors in {x l}. In an analogy to the TSP problem, this represents a sequence of d cities lying in an n-dim space. We are concerned with generating a binary adjacency matrix A ∈ {0, 1} d×d so that the corresponding graph is acyclic and achieves the best score. In this work we consider encoder-decoder models for graph generation: Encoder We use the attention based encoder in the Transformer structure proposed by. We believe that the self-attention scheme, together with structural DAG constraint, is capable of finding the causal relations amongst variables. Other attention based models such as graph attention network (Veličković et al., 2018) may also be used, which will be considered in a future work. Denote the output of the encoder by enc i with dimension d e, corresponding to each inputx i. Decoder Our decoder generates the graph adjacency matrix in an element-wise manner, by building relationships between two encoder outputs enc i and enc j. We consider the single layer decoder where are trainable parameters and d h is the hidden dimension associated with the decoder. To generate a binary adjacency matrix A, we pass each entry g ij into a logistic sigmoid function σ(·) and then sample according to a Bernoulli distribution with probability σ(g ij), which indicates the probability of existing an edge from x i to x j. To avoid self-loop, we simply mask the (i, i)-th entry in the adjacency matrix. Other possible decoder choices include the neural tensor network model from and the bilinear model to build the pairwise relationships between encoder outputs. Another choice is the Transformer structure which generates an adjacency matrix in a row-wise manner. Empirically, we find that the single layer decoder performs the best, possibly because it contains less parameters and is easier to train to find better DAGs, while the self-attention based encoder has provided sufficient interactions amongst the variables for causal discovery. In Appendix A, we provide more details on the above decoders and their empirical with linear-Gaussian data models. In this section, we propose to use RL as the search strategy for finding the DAG with the best score, outlined in Figure 1. As one will see, the proposed method improves the search ability over traditional score-based methods and also allows for flexible score functions under the acyclicity constraint. Score Function In this work, we consider existing score functions to construct the reward that will be maximized by an RL agent. Often score-based methods assume a parametric model for causal relationships (e.g., linear-Gaussian equations or multinomial distribution), which introduces a set of parameters θ. Among all score functions that can be directly included here, we focus on the BIC score that is not only consistent but also locally consistent for its decomposability . The BIC score for a given directed graph G is whereθ is the maximum likelihood estimator and d θ denotes the dimensionality of the parameter θ. We assume i.i.d. Gaussian additive noises throughout this paper. If we apply linear models to each causal relationship and letx k i be the corresponding estimate for x k i, the i-th entry in the k-th observed sample, then we have the BIC score being (up to some additive constant) where 2 denotes the residual sum of squares for the i-th variable. The first term in Eq. is equivalent to the log-likelihood objective used by GraN-DAG and the second term adds penalty on the number of edges in the graph G. Further assuming that the noise variances are equal (despite the fact that they may be different), we have We notice that i RSS i is the least squares loss used in NOTEARS . Besides assuming linear models, other regression methods can also be used to estimate x k i; in Section 6, we will use quadratic regression and like Gaussian Process Regression (GPR) to model causal relationships with the observed data for our experiments. Acyclicity A remaining issue is the acyclicity constraint. Other than GES that explicitly checks for acyclicity each time an edge is added, we add penalty terms w.r.t. acyclicity to the score function to enforce acyclicity in an implicit way and allow the generated graph to change more than one edges at each iteration. In this work, we use a recent from: a directed graph G with binary adjacency matrix A is acyclic if and only if where e A is the matrix exponential of A. We find that h(A), which is non-negative, can be small for certain cyclic graphs and its minimum over all non-DAGs is not easy to compute. Consequently, we would require a very large penalty weight to obtain exact DAGs if only h(A) is used. As such, we add another penalty term, the indicator function w.r.t. acyclicity, to induce exact DAGs. Here we remark that other functions, e.g., the total length all cyclic paths in the graph, which compute some'distance' from a directed graph to DAGs and need not be smooth, may also be used. Reward Our reward incorporates both the score function and the acyclicity constraint: where I(·) denotes the indicator function and λ 1, λ 2 ≥ 0 are two penalty parameters. It is not hard to see that the larger λ 1 and λ 2 are, the more likely a generated graph with a high reward is acyclic. We then aim to maximize the reward over all possible directed graphs, or equivalently, we have An interesting question is whether this new formulation is equivalent to the original problem with hard acyclicity constraint. Fortunately, the following proposition guarantees that Problems and are equivalent with properly chosen λ 1 and λ 2, which can be verified by showing that a minimizer of one problem is also a solution to the other. A proof is provided in Appendix B for completeness. Proposition 1. Let h min > 0 be the minimum of h(A) over all directed cyclic graphs, i.e., h min = min G / ∈DAGs h(A). Let S * denote the optimal score achieved by some DAG in Problem. Assume that S L ∈ R is a lower bound of the score function over all possible directed graphs, i.e., S L ≤ min G S(G), and S U ∈ R is an upper bound on the optimal score with S * ≤ S U. Then Problems and are equivalent if For practical use, we need to find respective quantities in order to choose proper penalty parameters. An upper bound S U can be easily found by drawing some random DAGs or using the from other methods like NOTEARS. A lower bound S L depends on the particular score function. With BIC score, we can fit each variable x i against all the rest variables, and use only the RSS i terms but ignore the additive penalty on the number of edges. With the independence based score function proposed by, we may simply set S L = 0. The minimum term h min, as previously mentioned, may not be easy to find. Fortunately, with λ 1 = S U − S L, Proposition 1 guarantees the equivalence of Problems and for any λ 2 ≥ 0. However, simply setting λ 2 = 0 could only get good performance with very small graphs (see a discussion in Appendix C). We therefore pick a relatively small value for λ 2, which helps to generate directed graphs that become closer to DAGs. Empirically, we find that if the penalty weights are set too large, the score function would have little effect on the reward, which limits the exploration of the RL agent and usually in DAGs with high scores. Similar to Lagrangian methods, we will start with small penalty weights and gradually increase them so that the condition in Proposition 1 is satisfied. Meanwhile, we notice that different score functions may have different ranges while the acyclicity penalty terms are independent of the Algorithm 1 The proposed RL approach to score-based causal discovery Require: score parameters: S L, S U, and S 0; penalty parameters: λ 1, ∆ 1, λ 2, ∆ 2, and Λ 2; iteration number for parameter update: t 0. 1: for t = 1, 2,... do Run actor-critic algorithm, with score adjustment by if the maximum reward corresponds to a DAG with score S min then 5: update recorded rewards according to new λ 1 and λ 2 9: end if 10: end for particular range of the score function. Therefore, we also adjust the predefined scores to a certain range by using S 0 (S −S L)/(S U −S L) for some S 0 > 0 and the optimal score will lie in [0, S 0]. 1 Our algorithm is summarized in Algorithm 1, where ∆ 1 and ∆ 2 are the updating parameters associated with λ 1 and λ 2, respectively, and t 0 denotes the updating frequency. The weight λ 2 is updated in a similar manner to the updating rule on the Lagrangian multiplier used by NOTEARS and we set Λ 2 as an upper bound on λ 2, as previously mentioned. In all our experiments that use BIC as score function, S L is obtained from a complete directed graph and S U is from an empty graph. Since S U with the empty graph can be very high for large graphs, we also update it by keeping track of the lowest score achieved by DAGs generated during training. Other parameter choices in this work are S 0 = 5, t 0 = 1000, λ 1 = 0, ∆ 1 = 1, λ 2 = 10 − d/3, ∆ 2 = 10 and Λ 2 = 0.01. We comment that these parameter choices may be further tuned for specific applications, and we can select the graph that is acyclic and achieves the best score, among all the final outputs (cf. Section 5.3) of the RL approach with different parameter choices. We believe that the exploitation and exploration scheme in the RL paradigm provides an appropriate way to guide the search. Let π(· | s) and ψ denote the policy and NN parameters for graph generation, respectively. Our training objective is the expected reward defined as During training, the input s is constructed by randomly drawing samples from the observed dataset X, as described in Section 4. We resort to policy gradient methods and stochastic methods to optimize the parameters ψ. The gradient ∇ ψ J(ψ | s) can be obtained by the well-known REINFORCE algorithm . We draw B samples s 1, s 2,..., s B as a batch to estimate the gradient which is then used to train the NNs through stochastic optimization methods like Adam . Using a parametric baseline to estimate the reward can also help training . For the present work, our critic is a simple 2-layer feed-forward NN with ReLU units, with the input being the encoder outputs {enc i}. The critic is trained with Adam on a mean squared error between its predictions and the true rewards. An entropy regularization term is also added to encourage exploration of the RL agent. Although policy gradient methods only guarantee local convergence , here we remark that the inferred graphs from the actor-critic algorithm are all DAGs in our experiments. Training an RL agent typically requires many iterations. In the present work, we find that computing the rewards for generated graphs is much more time-consuming than training NNs. Therefore, we record the computed rewards corresponding to different graph structures. Moreover, the BIC score can be decomposed according to single causal relationships and we also record the corresponding RSS i to avoid repeated computations. Since we are concerned with finding a DAG with the best score rather than a policy, we record all the graphs generated during the training process and output the one with the best reward. In practice, the graph may contain spurious edges and further processing is needed. To this end, we can prune the estimated edges in a greedy way, according to either the regression performance or the score function. For an inferred causal relationship, we remove a parental variable and calculate the performance of the ing graph, with all other causal relationships unchanged. If the performance does not degrade or degrade within a predefined tolerance, we accept pruning and continue this process with the pruned causal relationship. For linear models, pruning can be simply done by thresholding the estimated coefficients. Related to the above pruning process is to add to the reward an increased penalty weight on the number of edges of a given graph. However, this weight is not easy to choose, as a large weight may incur missing edges. In this work, we stick to the penalty weight log n that is included in the BIC score and then apply pruning to the inferred graph in order to reduce false discoveries. We report empirical on synthetic and real datasets to compare our approach against both traditional and recent gradient based approaches, including GES (with BIC score) , the PC algorithm (with Fisher-z test and p-value 0.01) , ICA-LiNGAM , the Causal Additive Model (CAM) based algorithm proposed by Bühlmann et al., NOTEARS , DAG-GNN , and GraN-DAG , among others. All these algorithms have available implementations and we give a brief description on these algorithms and their implementations in Appendix D. Default hyper-parameters of these implementations are used unless otherwise stated. For pruning, we use the same thresholding method for ICA-LiNGAM, NOTEARS, and DAG-GNN. Since the authors of CAM and GraN-DAG proposed to apply significance testing of covariates based on generalized additive models and then declare significance if the reported p-values are lower than or equal to 0.001, we stick to the same pruning method for CAM and GraN-DAG. The proposed RL based approach is implemented based on an existing Tensorflow implementation of neural combinatorial optimizer (see Appendix D for more details). The decoder is modified as described in Section 4 and the RL algorithm related hyper-parameters are left unchanged. We pick B = 64 as batch size at each iteration and d h = 16 as the hidden dimension with the single layer decoder. Our approach is combined with the BIC scores under Gaussianity assumption given in Eqs. and, and are denoted as RL-BIC and RL-BIC2, respectively. We evaluate the estimated graphs using three metrics: False Discovery Rate (FDR), True Positive Rate (TPR), and Structural Hamming Distance (SHD) which is the smallest number of edge additions, deletions, and reversals to convert the estimated graph into the true DAG. The SHD takes into account both false positives and negatives and a lower SHD indicates a better estimate of the causal graph. Since GES and PC may output unoriented edges, we follow to treat GES and PC favorably by regarding undirected edges as true positives as long as the true graph has a directed edge in place of the undirected edge. Given number of variables d, we generate a d × d upper triangular matrix as the graph binary adjacency matrix, in which the upper entries are sampled independently from Bern(0.5). We assign edge weights independently from Unif ([−2, −0.5] ∪ [0.5, 2]) to obtain a weight matrix W ∈ R d×d, and then sample x = W T x + n ∈ R d from both Gaussian and non-Gaussian noise models. The non-Gaussian noise is the same as the one used for ICA-LiNGAM , which generates samples from a Gaussian distribution and passes them through a power nonlinearity to make them non-Gaussian. We pick unit variances for all noises in both models and generate m = 5, 000 samples as our datasets. A random permutation of variables is then performed. This data generating procedure is similar to that used by NOTEARS and DAG-GNN and the true causal graphs in both cases are known to be identifiable (Peters and Bühlmann, 2013;). We first consider graphs with d = 12 nodes. We use n = 64 for constructing the input sample and set the maximum number of iterations to 20, 000. Figure 2 shows the learning process of the proposed method RL-BIC2 on a linear-Gaussian dataset. We use a threshold 0.3, same as NOTEARS and DAG-GNN with this data model, to prune the estimated edges. In this example, RL-BIC2 generates 683, 784 different graphs during training, much lower than the total number (around 5.22 × 10 26) of DAGs. The pruned DAG turns out to be exactly the same as the underlying causal graph. We report the empirical on LiNGAM and linear-Gaussian data models in Table 1. Both PC and GES perform poorly, possibly because we consider relatively dense graphs for our data generating procedure. CAM does not perform well either, as it assumes nonlinear causal relationships. ICA-LiNGAM recovers all the true causal graphs for LiNGAM data but performs poorly on linearGaussian data. This is not surprising because ICA-LiNGAM works for non-Gaussian noise and does not provide guarantee for linear-Gaussian datasets. Both NOTEARS and DAG-GNN have good causal discovery whereas GraN-DAG performs much worse. We believe that it is because GraN-DAG uses 2-layer feed-forward NNs to model the causal relationships, which may not be able to learn a good linear relationship in this experiment. Modifying the feed-forward NNs to linear functions reduces to NOTEARS with negative log-likelihood as loss function, which yields similar performance on these datasets (see Appendix E.1 for detailed ). As to our proposed methods, we observe that RL-BIC2 recovers all the true causal graphs on both data models in this experiment while RL-BIC has a worse performance. One may wonder whether this observation is due to the same noise variances that are used in our data models; we conduct additional experiments where the noise variances are randomly sampled and RL-BIC2 still outperforms RL-BIC by a large margin (see also Appendix E.1). Nevertheless, with the same BIC score, RL-BIC performs much better than GES on both datasets, indicating that the RL approach brings in a greatly improved search ability. Finally, we test the proposed method on larger graphs with d = 30 nodes, where the upper entries are sampled independently from Bern(0.2). This edge probability choice corresponds to the fact that large graphs in practice usually has a lower average degree; see, e.g., the experiment settings of;;. To incorporate this prior information in our approach, we add to each g ij a common bias term initialized to −10.0 (see Appendix E.1 for details). Considering the much increased search space, we also choose a larger number of observed samples, n = 128, to construct the input for graph generator and increase the training iterations to 40, 000. On LiNGAM datasets, RL-BIC2 has FDR, TPR, and SHD being 0.14 ± 0.15, 0.94 ± 0.07, and 19.8 ± 23.0, respectively, comparable to NOTEARS with 0.13 ± 0.09, 0.94 ± 0.04 and 17.2 ± 13.12. We now consider nonlinear causal relationships with quadratic functions. We generate an upper triangular matrix in a similar way to the first experiment. For a causal relationship with parents T at the i-th node, we expand x pa(i) to contain both first-and second-order features. The coefficient for each term is then either 0 or sampled from Unif ([−1, −0.5] ∪ [0.5, 1]), with equal probability. If a parent variable does not appear in any feature term with a non-zero coefficient, then we remove the corresponding edge in the causal graph. The rest follows the same as in first experiment and here we use the non-Gaussian noise model with 10-node graphs and 5, 000 samples. The true causal graph is identifiable according to. For this quadratic model, there may exist very large variable values which cause computation issues for quadratic regression. We treat these samples as outliers and detailed processing is given in Appendix E.2. We use quadratic regression for a given causal relationship and calculate the BIC score (assuming equal noise variances) in Eq.. For pruning, we simply apply thresholding, with threshold as 0.3, to the estimated coefficients of both first-and second-order terms. If the coefficient of a second-order term, e.g., x i1 x i2, is non-zero after thresholding, then we have two directed edges that are from x i1 to x i and from x i2 to x i, respectively. We do not consider PC and GES in this experiment due to their poor performance in the first experiment. Our with 10-node graphs are reported in Table 2, which shows that RL-BIC2 achieves the best performance. For fair comparison, we apply the same quadratic regression based pruning method to the outputs of NOTEARS, denoted as NOTEARS-2. We see that this pruning further reduces FDR, i.e., removes spurious edges, with little effect on TPR. Since pruning does not help discover additional positive edges or increase TPR, we will not apply this pruning method to other methods as their TPRs are much lower than that of RL-BIC2. Finally, with prior knowledge that the function form is quadratic, we can modify NOTEARS to apply quadratic functions to modeling the causal relationships, with an equivalent weighted adjacency matrix constructed using the coefficients of the first-and second-order terms, similar to the idea used by GraN-DAG (detailed derivations are given in Appendix E.2). The problem then becomes a nonconvex optimization problem with (d − 1)d 2 /2 parameters (which are the coefficients of both first-and second-order features), compared to the original problem with d 2 parameters. This method corresponds to NOTEARS-3 in Table 2. Despite the fact that NOTEARS-3 did not achieve a better overall performance than RL-BIC2, we comment that it discovered almost correct causal graphs (with SHD ≤ 3) on more than half of the datasets, but performed poorly on the rest datasets. We believe that it is due to the increased number of optimization parameters and the more complicated equivalent adjacency matrix which make the optimization problem harder to solve. Meanwhile, we do not exclude that NOTEARS-3 can achieve a better causal discovery performance with other optimization algorithms. Given a randomly generated causal graph, we consider another nonlinear model where each causal relationship f i is a function sampled from a Gaussian process, with RBF kernel of bandwidth one. The additive noise n i is normally distributed with variance sampled uniformly. This setting is known to be identifiable according to. We use a setup that is also considered by GraN-DAG : 10-node and 40-edge graphs with 1, 000 generated samples. The empirical are reported in Table 3. One can see that ICA-LiNGAM, NOTEARS, and DAG-GNN perform poorly on this data model. A possible reason is that they may not be able to model this type of causal relationship. More importantly, these methods operate on the weighted adjacency matrix which is not obvious here. For our method, we apply Gaussian Process Regression (GPR) with RBF kernel to model the causal relationships. Notice that even though the observed data are from a function sampled from Gaussian process, it is not guaranteed that GPR with the same kernel can achieve a good performance. Indeed, using a fixed kernel bandwidth would lead to severe overfitting that incurs many spurious edges and the graph with the highest reward is usually not a DAG. To proceed, we normalize the observed data and apply median heuristics for kernel bandwidth. Both our methods perform reasonably well, with RL-BIC outperforming all the other methods. We consider a real dataset to discover a protein signaling network based on expression levels of proteins and phospholipids . This dataset is a common benchmark in graphical models, with experimental annotations well accepted by the biological community. Both observational and interventional data are contained in this dataset. Since we are interested in using observational data to infer causal mechanisms, we only consider the observational data with m = 853 samples. The ground truth causal graph given by has 11 nodes and 17 edges. Notice that the true graph is indeed sparse and an empty graph can have an SHD as low as 17. Therefore, we report more detailed regarding the estimated graph: number of total edges, number of correct edges, and the SHD. Both PC and GES output too many unoriented edges, and we will not report their here. We apply GPR with RBF kernel to modeling the causal relationships, with the same data normalization and median heuristics for kernel bandwidth as in Section 6.3. We also use CAM pruning on the inferred graph from the training process. The empirical are given in Table 4. Both RL-BIC and RL-BIC2 achieve promising , compared with other methods. We have proposed to use RL to search for the DAG with the optimal score. Our reward is designed to incorporate a predefined score function and two penalty terms to enforce acyclicity. We use the actor-critic algorithm as our RL algorithm, where the actor is constructed based on recently developed encoder-decoder models. Experiments are conducted on both synthetic and real datasets to show the advantages of our method over other causal discovery methods. We have also shown the effectiveness of the proposed method with 30-node graphs, yet dealing with large graphs (with more than 50 nodes) is still challenging. Nevertheless, many real applications, like Sachs dataset , have a relatively small number of variables. Furthermore, it is possible to decompose large causal discovery problems into smaller ones; see, e.g.,. Prior knowledge or constraint-based methods is also applicable to reduce the search space. There are several future directions from the present work. In our current implementation, computing scores is much more time consuming than training NNs. While more computing resources definitely help accelerate training, we believe that developing an efficient and effective score function is key to the proposed approach. Many other RL algorithms may also be used. For example, the asynchronous advantage actor-critic algorithm has been shown to be effective in many applications . In addition, we observe that the total iteration numbers used in our experiments are usually more than needed. A proper early stopping criterion will be favored. We briefly describe the NN based decoders for generating binary adjacency matrices: • Single layer decoder: where are trainable parameters and d h denotes the hidden dimension associated with the decoder. • Bilinear decoder: where W ∈ R de×de is a trainable parameter. • Neural Tensor Network (NTN) decoder :, and b ∈ R K×1. • Transformer decoder uses the multi-head attention module to obtain the decoder outputs {dec i}, followed by a feed-forward NN whose weights are shared across all dec i . The output consists of d vectors in R d which are treated as the row vectors of a d × d matrix. We then pass each element of this matrix into sigmoid functions and sample a binary adjacency matrix accordingly. Table 5 provides the empirical on linear-Gaussian data models with 12-node graphs and unit variances (see Section 6.1 for more details on this data generating procedure). In our implementation, we pick d e = 64 as the dimension of the encoder output, d h = 16 for the single layer decoder, and K = 2 for the NTN decoder. We find that single layer decoder performs the best, possibly because it has less parameters and is easier to train to find better DAGs, while the Transformer encoder has provided sufficient interactions amongst variables to capture the causal relationships. LinearGaussian FDR 0 ± 0 0.07 ± 0.13 0.12 ± 0.14 0.67 ± 0.07 TPR 1 ± 0 0.95 ± 0.08 0.90 ± 0.11 0.32 ± 0.09 SHD 0 ± 0 4.00 ± 7.04 7.40 ± 8.52 38.2 ± 3.12 B EQUIVALENCE OF PROBLEMS AND Proof of Proposition 1. Let G be a solution to Problem. Then we have S * = S(G) and G must be a DAG due to the hard acyclicity constraint. Assume that G is not a solution to Problem, which indicates that there exists a directed graph G (with binary adjacency matrix A) so that Clearly, G cannot be a DAG, for otherwise we would have a DAG that achieves a lower score than the minimum S *. By our assumption, it follows that which contradicts the fact that S U is an upper bound on S *. For the other direction, let G be a solution to Problem but not a solution to Problem. This indicates that either G is not a DAG or G is a DAG but has a higher score than the minimum score, i.e., S(G) > S *. The latter case clearly contradicts the definition of the minimum score. For the former case, assume that some DAG G achieves the minimum score. Then plugging G into the negative reward, we can get the same inequality in Eq. since both penalty terms are zeros for a DAG. This then contradicts the assumption that G minimizes the negative reward. Although setting λ 2 = 0, or equivalently using only the indicator function w.r.t. acyclicity, can still make Problem equivalent to the original problem with hard acyclicity constraint, we remark that this choice usually does not in good performance of the RL approach, largely due to that the reward with only the indicator term is likely to fail to guide the RL agent to generate DAGs. To see why it is the case, consider two cyclic directed graphs, one with all the possible directed edges in place and the other with only two edges (i.e., x i → x j and x j → x i for some i = j). The latter is much'closer' to acyclicity in many senses, such as h(A) given in Eq. and number of edge deletion, addition, and reversal to make a directed graph acyclic. Assume a linear data model that has a relatively dense graph, the former will have a lower BIC score when using linear regression for fitting causal relations, yet the penalty terms of acyclicity are the same with only the indicator function. The former graph then has a higher reward, which does not help the agent to tend to generate DAGs. Also notice that the graphs in our approach are generated according to Bernoulli distributions determined by NN parameters that are randomly initialized. Without loss of generality, consider that each edge is drawn independently according to Bern(0.5). For small graphs (with less than or equal to 6 nodes or so), a few hundreds of samples of directed graphs are very likely to contain a DAG. Yet for large graphs, the probability of sampling a DAG is much lower. If no DAG is generated during training, the RL agent can hardly learn to generate DAGs. The above facts indeed motivate us to choose a small value of λ 2 so that the agent can be trained to produce graphs closer to acyclicity and finally to generate exact DAGs. A question is then what if the RL approach starts with a DAG, e.g., by initializing the probability of generating each edge to be nearly zero. This setting did not lead to good performance, either. The generated directed graphs at early iterations can be very different from the true graphs in that many true edges do not exist, and the ing score is much higher than the minimum under the DAG constraint. With small penalty weights of the acyclicity terms, the agent could be trained to produce cyclic graphs with lower scores, similar to the case with randomly initialized NN parameters. On the other hand, large penalty weights, as we have discussed in the paper, limit exploration of the RL agent and usually in DAGs whose scores are far from optimum. We use existing implementations of causal discovery algorithms in comparison, listed below: • ICA-LiNGAM assumes linear non-Gaussian additive model for data generating procedure and applies independent component analysis method to recover the weighted adjacency matrix, followed by thresholding on the weights before outputting the inferred graph. A Python implementation is available at the first author's website https://sites.google.com/site/sshimizu06/lingam. • GES and PC: we use the fast greedy search implementation of GES which has been reported to outperform other techniques such as max-min hill climbing . Implementations of both methods are available through the py-causal package at https://github.com/bd2kccd/py-causal, written in highly optimized Java codes. • CAM decouples the causal order search among the variables from feature or edge selection in a DAG. CAM also assumes additive noise as in our work, with an additional condition that each function is nonlinear. Codes are available through the CRAN R package repository at https://cran.r-project.org/web/packages/CAM. • NOTEARS recovers the causal graph by estimating the weighted adjacency matrix with the least squares loss and the smooth characterization for acyclicity constraint, followed by thresholding on the estimated weights. Codes are available at the first author's github repository https://github.com/xunzheng/notears. We also re-implement the augmented Lagrangian method following the same updating rule on the Lagrange multiplier and the penalty parameter in Tensorflow, so that the augmented Lagrangian at each iteration can be readily minimized without the need of obtaining closedform gradients. We use this implementation in Sections 6.2 and 6.3 when the objective function and/or the acyclicity constraint are modified. • DAG-GNN formulates causal discovery in the framework of variational autoencoder, where the encoder and decoder are two shallow graph NNs. With a modified smooth characterization on acyclicity, DAG-GNN optimizes a weighted adjacency matrix with the evidence lower bound as loss function. Python codes are available at the first author's github repository https://github.com/fishmoon1234/DAG-GNN. • GraN-DAG uses feed-foward NNs to model each causal relationship and chooses the sum of all product paths between variables x i and x j as the (i, j)-th element of an equivalent weighted adjacency matrix. GraN-DAG uses the same smooth constraint from to find a DAG that maximizes the log-likelihood of the observed samples. Codes are available at the first author's github repository https: //github.com/kurowasan/GraN-DAG. Our implementation is based on an existing Tensorflow implementation of neural combinatorial optimizer, which is available at https://github.com/MichelDeudon/ neural-combinatorial-optimization-rl-tensorflow. We add an entropy regularization term, and modify the reward and decoder as described in Section 4 and Section 5.1. E.1 EXPERIMENT 1 IN SECTION 6.1 We replace the feed-forward NNs with linear functions in GraN-DAG and obtain similar experimental as NOTEARS (FDR, TPR, SHD): 0.05 ± 0.04, 0.93 ± 0.06, 3.2 ± 2.93 and 0.05 ± 0.04, 0.95 ± 0.03, 2.40 ± 1.85 for LiNGAM and linear-Gaussian data models, respectively. We conduct additional experiments with linear models where the noise variances are uniformly sampled according to Unif ([0.5, 2] ). Results are given in Table 6. Table 6: Empirical on LiNGAM and linear-Gaussian data models with 12-node graphs and different noise variances. RL-BIC RL-BIC2 PC GES ICA-LiNGAM CAM NOTEARS DAG-GNN GraN-DAG LiNGAM FDR 0.29 ± 0.12 0.09 ± 0.06 0.57 ± 0.10 0.59 ± 0.13 0 ± 0 0.70 ±0.07 0.08 ± 0.10 0.14 ± 0.07 0.71 ± 0.10 TPR 0.77 ± 0.15 0.94 ± 0.03 0.28 ± 0.06 0.27 ± 0.10 1 ± 0 0.45 ± 0.12 0.94 ± 0.07 0.91 ± 0.04 0.25 ± 0.09 SHD 14.4 ± 7.17 4.00 ± 2.61 30.4 ± 4.13 32.0 ± 5.18 0 ± 0 41.6 ± 3.32 3.20 ± 3.97 6.60 ± 1.02 38.7 ± 4.86 LinearGaussian FDR 0.36 ± 0.07 0.10 ± 0.07 0.54 ± 0.10 0.61 ± 0.14 0.67 ± 0.05 0.65 ± 0.10 0.07 ± 0.09 0.12 ± 0.04 0.71 ± 0.12 TPR 0.68 ± 0.09 0.93 ± 0.04 0.29 ± 0.05 0.26 ± 0.11 0.75 ± 0.06 0.51 ± 0.14 0.95 ± 0.06 0.94 ± 0.04 0.21 ± 0.08 SHD 18.8 ± 3.43 4.60 ± 3.07 30.0 ± 2.83 32.2 ± 5.42 49.0 ± 4.82 37.8 ± 6.31 3.00 ± 3.58 5.40 ± 2.06 39.6 ± 5.85 Knowing a sparse true causal graph a priori is also helpful. To incorporate this information in our experiment with 30-node graphs, we add an additional biased termc ∈ R to each decoder output: for the single layer decoder, we have g ij (W 1, W 2, u) = u T tanh(W 1 enc i + W 2 enc j) +c, where we letc be trainable and other parameters have been defined in Appendix A. In our experiments, c is initialized to be −10; this choice aims to set a good starting point for generating graph adjacency matrices, motivated by the fact that a good starting point is usually helpful to locally convergent algorithms.

Represent the TLDR for finding the corresponding scientific article

We apply reinforcement learning to score-based causal discovery and achieve promising results on both synthetic and real datasets

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We propose a new class of inference models that iteratively encode gradients to estimate approximate posterior distributions.


Question: Represent the scientific article to find a short TLDR

Question: The topic modeling discovers the latent topic probability of given the text documents. To generate the more meaningful topic that better represents the given document, we proposed a universal method which can be used in the data preprocessing stage. The method consists of three steps. First, it generates the word/word-pair from every single document. Second, it applies a two way parallel TF-IDF algorithm to word/word-pair for semantic filtering. Third, it uses the k-means algorithm to merge the word pairs that have the similar semantic meaning. Experiments are carried out on the Open Movie Database (OMDb), Reuters Dataset and 20NewsGroup Dataset and use the mean Average Precision score as the evaluation metric. Comparing our with other state-of-the-art topic models, such as Latent Dirichlet allocation and traditional Restricted Boltzmann Machines. Our proposed data preprocessing can improve the generated topic accuracy by up to 12.99\%. How the number of clusters and the number of word pairs should be adjusted for different type of text document is also discussed. After millennium, most collective information are digitized to form an immense database distributed across the Internet. Among all, text-based knowledge is dominant because of its vast availability and numerous forms of existence. For example, news, articles, or even Twitter posts are various kinds of text documents. For the human, it is difficult to locate one's searching target in the sea of countless texts without a well-defined computational model to organize the information. On the other hand, in this big data era, the e-commerce industry takes huge advantages of machine learning techniques to discover customers' preference. For example, notifying a customer of the release of "Star Wars: The Last Jedi" if he/she has ever purchased the tickets for "Star Trek Beyond"; recommending a reader "A Brief History of Time" from Stephen Hawking in case there is a "Relativity: The Special and General Theory" from Albert Einstein in the shopping cart on Amazon. The content based recommendation is achieved by analyzing the theme of the items extracted from its text description. Topic modeling is a collection of algorithms that aim to discover and annotate large archives of documents with thematic information BID0. Usually, general topic modeling algorithms do not require any prior annotations or labeling of the document while the abstraction is the output of the algorithms. Topic modeling enables us to convert a collection of large documents into a set of topic vectors. Each entry in this concise representation is a probability of the latent topic distribution. By comparing the topic distributions, we can easily calculate the similarity between two different documents. BID25 Some topic modeling algorithms are highly frequently used in text-mining BID13, preference recommendation BID27 and computer vision BID28. BID0 Many of the traditional topic models focus on latent semantic analysis with unsupervised learning. Latent Semantic Indexing (LSI) BID11 applies Singular-Value Decomposition (SVD) BID6 to transform the term-document matrix to a lower dimension where semantically similar terms are merged. It can be used to report the semantic distance between two documents, however, it does not explicitly provide the topic information. The Probabilistic Latent Semantic Analysis (PLSA) BID9 model uses maximum likelihood estimation to extract latent topics and topic word distribution, while the Latent Dirichlet Allocation (LDA) BID1 model performs iterative sampling and characterization to search for the same information. The availability of many manually categorized online documents, such as Internet Movie Database (IMDb) movie review Inc., Wikipedia articles, makes the training and testing of topics models possible. All of the existing workds are based on the bag-of-words model, where a document is considered as a collection of words. The semantic information of words and interaction among objects are assumed to be unknown during the model construction. Such simple representation can be improved by recent research advances in natural language processing and word embedding. In this paper, we will explore the existing knowledge and build a topic model using explicit semantic analysis. The work studies the best data processing and feature extraction algorithms for topic modeling and information retrieval. We investigate how the available semantic knowledge, which can be obtained from language analysis or from existing dictionary such as WordNet, can assist in the topic modeling. Our main contributions are:• We redesign a new topic model which combines two types of text features to be the model input.• We apply the numerical statistic algorithm to determine the key elements for each document dynamically.• We apply a vector quantization method to merge and filter text unit based on the semantic meaning.• We significantly improve the accuracy of the prediction using our proposed model. The rest of the paper is structured as follows: In Section 2, we review the existing methods, from which we got the inspirations. This is followed in Section 3 by details about our topic models. Section 4 describes our experimental steps and evaluate the . Finally, Section 5 concludes this work. Many topic models have been proposed in the past decades. This includes LDA, Latent Semantic Analysis(LSA), word2vec, and Restricted Boltzmann Machine (RBM), etc. In this section, we will compare the pros and cons of these topic models. LDA was one of the most widely used topic models. LDA introduces sparse Dirichlet prior distributions over document-topic and topic-word distributions, encoding the intuition that documents cover a small number of topics and that topics often use a small number of words. BID1 LSA was another topic modeling technique which is frequently used in information retrieval. LSA learned latent topics by performing a matrix decomposition (SVD) on the term-document matrix. BID4 In practice, training the LSA model is faster than training the LDA model, but the LDA model is more accurate than the LSA model. Traditional topic models did not consider the semantic meaning of each word and cannot represent the relationship between different words. Word2vec can be used for learning high-quality word vectors from huge data sets with billions of words, and with millions of words in the vocabulary. BID14 During the training, the model generated word-context pairs by applying a sliding window to scan through a text corpus. Then the word2vec model trained word embeddings using word-context pairs by using the continuous bag of words (CBOW) model and the skip-gram model. BID15 The generated word vectors can be summed together to form a semantically meaningful combination of both words. Moreover, there were a lot of extensions of the word2vec model, such as paragraph2vec and LDA2vec. Paragraph2vec was an unsupervised framework that learned continuous distributed vector representations for pieces of texts. The training of the paragraph2vec model is based on the similar idea as the word2vec. One of the advantages of the generated paragraph vector was that they take into consideration the word order. BID12 LDA2vec focused on utilizing document-wide feature vectors while simultaneously learning continuous document weights loading onto topic vectors. LDA2vec embedded both words and document vectors into the same space and train both representations simultaneously. BID18 RBM was proposed to extract low-dimensional latent semantic representations from a large collection of documents BID8. The architecture of the RBMs is an undirected bipartite graphic, in which word-count vectors were modeled as Softmax input units and the output units were usually binary units. The RBM model can be generalized much better than LDA in terms of both log-probability on the document and the retrieval accuracy. A deeper structure of neural network was developed based on stacked RBMs, which was the Deep Belief Network (DBN). In BID7, the input layer was the same as RBM mentioned above, other layers are all binary units. 3.1 WORD PAIR BASED RBM MODEL Current RBM model for topic modeling uses Bag of Words approach. Each visible neuron represents the number of appearance of a dictionary word. We believe that the order of the words also exhibits rich information, which is captured by the bag of words approach. Our hypothesis is that including word pairs (with specific dependencies) helps to improve topic modeling. In this work, Stanford natural language parser BID3 BID19 is used to analyze sentences in both training and testing texts, and extract word pairs. For example, if we have the following sentence, by applying the word pair extraction, we will get 38 word pairs and one root pair as the FIG0.The strongest rain ever recorded in India shut down the financial hub of Mumbai, snapped communication lines, closed airports and forced thousands of people to sleep in their offices or walk home during the night, officials said today. The first part in each word pair segment represents the relationship between two words, such as det, nsubj, advmod, etc. And the number after each word gives its position of in the original sentence. The two words extracted in this way are not necessarily adjacent to each other, however, they are semantically related. Because each single word may have different combinations with other words, the total number of the word pairs will be much larger than the number of word in the training dataset. If we use all word pairs which are extracted from the training dataset, it will significantly increase the size of our dictionary and reduce the performance. So, we only keep the first 10000 most frequent word pairs to be our word pair dictionary. TF-IDF stands for the term frequency -inverse document frequency, and the TF-IDF weight is a weight often used in information retrieval and text mining. This weight is a statistical measure used to evaluate how important a word is to a document in a collection or corpus. The importance increases proportionally to the number of times a word appears in the document but is offset by the frequency of the word in the corpus. BID23 BID22 BID21 BID29 The equation of the TF-IDF is as following: DISPLAYFORM0 N umber of times term t appears in a docuemnt T otal number of terms in the document DISPLAYFORM1 DISPLAYFORM0 Term Frequency (TF), Equation 1, measures how frequently a term occurs in a document. Inverse document frequency (IDF), Equation 2, measures how important a term is. There are many words in the training dataset. If we use all words to build the training dictionary, it will contain a lot of high frequency but useless words, like "first" and "name". Beside removing stop word and processing word tense in the dataset, we also used the TF-IDF algorithm to filter the dataset. In addition, based on the original TF-IDF algorithm, we proposed a two-step TF-IDF processing method. This method is applied to keep the number of word pairs to a manageable size as shown in First, word pairs are generated and word-level TF-IDF is performed. The of word level TF-IDF is used as a filter and a word pair is kept only if the TF-IDF scores of both words are higher than the threshold (0.01). After that, we treat each word pair as a single unit, and the TF-IDF algorithm is applied to the word pairs and further filter out word pairs that are either too common or too rare. Even with the TF-IDF processing, the size of the word pair dictionary is still prohibitively large and selecting only the most frequent ones is a brute force approach. We further cluster semantically close word pairs to reduce the dictionary size. The semantic distance between two words is measured as the distance of their embedding vectors calculated using Googles word2vec model. The only issue is how to determine the cluster centrum. Our first approach is to use upper level words in WordNet BID17 BID16 as cluster centrum. Our hypothesis is that, since those upper-level words have broader meaning, a relatively small number of these words can provide good coverage of the semantical space. We first build a word level tree based on the relations specified in the WordNet. We then pick those words that are closest to the root as the cluster centrum. Words usually have multiple paths to the root. We first add those synsets that have only one path to the root into the graph, then iteratively examine the concepts that we have left and add the paths that grow the tree by as little as possible. Based on which path is selected, this approach can be further divided into min-path clustering and max-path clustering. The min-path clustering will add the shortest path and the max-path clustering will add the longest path to the graph The second approach is K-means clustering. By applying the K-mean algorithm, we group the embedding vector of words into K clusters. Then we use the index of each cluster to represent a group of words. Because the word embedding is a very dense space, the k-mean clustering does not give good Silhouette score BID20. Our original hypothesis is that the K-mean clustering based approach will not be as effective as the WordNet-based approach. However, our experimental show that this is not right. So, we pick the K-means clustering algorithm to be our final feature dictionary organization method. In our experiment, we generate the topic distribution for each document by using RBM model. Then we retrieve the top N documents by calculating Euclidean distance. Our proposed method is evaluated on 3 datasets: OMDb, Reuters, and 20NewsGroup. For the Reuters and 20NewsGroup dataset, we download them from BID2. The OMDb dataset is collected manually by using OMDb APIs (Fritz). All the datasets are divided into three sub-dataset: training, validation, and testing. The split ratio is 70:10:20. For each dataset, a 5-fold cross-validation is applied.• OMDb stands for The Open Movie Database. The training dataset contains 6043 movie descriptions, the validation dataset contains 863 movie descriptions and the testing dataset contains 1727 movie descriptions. We define the class for each movie by applying K-means clustering on category information. The value of K is set to 20.• Reuters, these documents appeared on the Reuters newswire in 1987 and were manually classified by personnel from Reuters Ltd. There are 7674 documents in total which belongs to 8 classes. The training dataset contains 5485 news, the validation dataset contains 768 news and the testing dataset contains 1535 news.• 20NewsGroup, this dataset is a collection of approximately 20,000 newsgroup documents, partitioned (nearly) evenly across 20 different newsgroups. The training dataset contains 13174 news, the validation dataset contains 1882 news and the testing dataset contains 3765 news. We use mean Average P recision (mAP) score to evaluate our proposed method. It is a score to evaluate the information retrieval quality. This evaluation method considers the effect of order in the information retrieval . If the relational is shown in the front position, the score will tend to 1; if the relational is shown in the back position, the score will tend to 0. mAP 1, mAP 3, mAP 5, and mAP 10 are used to evaluate the retrieval performance. For each document, we retrieve 1, 3, 5, and 10 documents whose topic vectors have the smallest Euclidean distance with that of the query document. The documents are considered as relevant if they share the same class label. Before we calculate the mAP, we need to calculate the Average P recision (AveP) for each document first. The equation of AveP is described below. DISPLAYFORM0 where rel(k) is an indicator function equaling 1 if the item at rank k is a relevant document, 0 otherwise. BID26 Note that the average is over all relevant documents and the relevant documents not retrieved get a precision score of zero. The equation of the mean Average P recision (mAP) score is as following: DISPLAYFORM1 where Q indicates the total number of queries. In this experiment, the total feature size is a fixed number. Then, we compare the performance between two RBM models. One of them only considers words as the input feature, while the other has combined words and word pairs as the input feature. The total feature size varies from 10500, 11000, 11500, 12000, 12500, 15000. For the word/word pair combined RBM model, the number of word is 10000, and the number of word pairs is varied to meet the total feature requirement. Both models are applied to the OMDb dataset, and the are shown in FIG2 and TAB0, the word/word pair combined model almost always performs better than the word-only model. For the mAP 1, the mAP 5 and the mAP 10, the most significant improvement shown in Feature Number = 11000, about 10.48%, 7.97% and 9.83%. For the mAP 3, the most significant improvement shown in Feature Number = 12000, about 9.35%. The two models are further applied on the Reuters dataset, and the are shown in FIG3 and TAB1. Again the word/word pair combined model outperforms the word-only model almost all the time. For the mAP 1, the most significant improvement happens when Feature Number = 12500. Under this feature size, the combined model improves the mAP score by approximately 1.05%. For the mAP 3, the mAP 5 and the mAP 10, the most significant improvement happens when Feature Number = 15000. Under this feature size, the combined model gives about 1.11%, 1.02% and 0.89% improvement. The for 20NewsGroup dataset are shown in Figure 5 and Table 3. Similar to previous two datasets, all the from word/word pair combined model are better than the word-only model. For the mAP 1 the mAP 3, the mAP 5 and the mAP 10, the most significant improvement happens when Feature Number = 11500. And the improvements are about 10.40%, 11.91%, 12.46% and 12.99%.Observing the for all three datasets, we found that they all reflect the same pattern. The word/word pair combined model consistently outperforms the word only model, given that both of them are constructed based on the same number of input features. In the second experiment, we focus on how the different K values affect the effectiveness of the generated word pairs in therms of their ability of topic modeling. First, we average the performance word/word pair combination all K value. The potential K values are 100, 300, 500, 800 and 1000. Then we compare the mAP between our model and the baseline model, which consists of word only input features. The OMDb dataset are shown in Table 4. As we can observe, all the K values give us better performance than the baseline. The most significant improvement shown in K = 100, which are about 2.41%, 2.15%, 1.46% and 4.46% for mAP 1, 3, 5, and 10 respectively. The of Reuters dataset are shown in FIG5 and Table 5. When the K value is greater than 500, all mAP for word/word pair combination model are better than the baseline. Because the mAP score for Reuters dataset in original model is already very high almost all of them higher than 0.9, it is hard to get the improvement as large as OMDb dataset. For the mAP 1, the most significant improvement happens when K = 500, which is 0.31%. For the mAP 5, the mAP 5 and the mAP 10, the most significant improvement shown in K = 800, about 0.50%, 0.38% and 0.42%.The for 20NewsGroup dataset are shown in FIG6 and Table 6. Similar to the Reuters dataset, when the K value is greater than 800, all mAP score for word/word pair combination model are better than the baseline. For the mAP 1, 3, 5, and 10, the most significant im- provements about 2.82%, 2.90%, 3.2% and 3.33% respectively, and they all happened when K = 1000.In summary, a larger K value can give us a better . As we can see from the Reuters dataset and the 20NewsGroup dataset, when K is greater than 800, the combined model outperforms the baseline model in all four mAP evaluation scores. However, the best appear when K = 800 or K = 1000. For the OMDb dataset, because the mAP score for the baseline model is not very high, the combined model gives better than the baseline model with any K value that we tested. In this experiment, we compare different word pair generation algorithms with the baseline. Similar to previous experiments, the baseline is the word-only RBM model whose input consists of the 10000 most frequent words. The "semantic" word pair generation is the method we proposed in this paper. By applying the idea from the skip-gram BID15 algorithm, we generate the word pairs from each word's adjacent neighbor, and we call it "N-gram" word pair generation. And the window size we used in here is N = 2. For the Non-K word pair generation, we use the same algorithm as the semantic except that no K-means clustering is applied on the generated word pairs. The first thing we observe from From the TAB2 is that both "semantic" word pair generation and "Non-K" word pair generation give us better mAP score than the baseline; however, the mAP score of the semantic generation is slightly higher than the mAP score of the Non-K generation. This is because, although both Non-K and semantic techniques extract word pairs using natural language processing, without the K-means clustering, semantically similar pairs will be considered separately. Hence there will be lots of redundancies in the input space. This will either increase the size of the input space, or, in order to control the input size, reduce the amount of information captured by the input set. The K-mean clustering performs the function of compress and feature extraction. The second thing that we observe is that, for the N-gram word pair gener-ation, its mAP score is even lower than the baseline. Beside the OMDb dataset, other two datasets show the same pattern. This is because the semantic model extracts word pairs from natural language processing, therefore those word pairs have the semantic meanings and grammatical dependencies. However, the N-gram word pair generation simply extracts words that are adjacent to each other. When introducing some meaningful word pairs, it also introduces more meaningless word pairs at the same time. These meaningless word pairs act as noises in the input. Hence, including word pairs without semantic importance does not help to improve the model accuracy. In this paper, we proposed a few techniques to processes the dataset and optimized the original RBM model. During the dataset processing part, first, we used a semantic dependency parser to extract the word pairs from each sentence of the text document. Then, by applying a two way parallel TF-IDF processing, we filtered the data in word level and word pair level. Finally, Kmeans clustering algorithm helped us merge the similar word pairs and remove the noise from the feature dictionary. We replaced the original word only RBM model by introducing word pairs. At the end, we showed that proper selection of K value and word pair generation techniques can significantly improve the topic prediction accuracy and the document retrieval performance. With our improvement, experimental have verified that, compared to original word only RBM model, our proposed word/word pair combined model can improve the mAP score up to 10.48% in OMDb dataset, up to 1.11% in Reuters dataset and up to 12.99% in the 20NewsGroup dataset.

represent the short summary for finding the corresponding scientific article (long)

We proposed a universal method which can be used in the data preprocessing stage to generate the more meaningful topic that better represents the given document

Represent the TLDR for finding the corresponding scientific article

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Multi-task learning has been successful in modeling multiple related tasks with large, carefully curated labeled datasets. By leveraging the relationships among different tasks, multi-task learning framework can improve the performance significantly. However, most of the existing works are under the assumption that the predefined tasks are related to each other. Thus, their applications on real-world are limited, because rare real-world problems are closely related. Besides, the understanding of relationships among tasks has been ignored by most of the current methods. Along this line, we propose a novel multi-task learning framework - Learning To Transfer Via Modelling Multi-level Task Dependency, which constructed attention based dependency relationships among different tasks. At the same time, the dependency relationship can be used to guide what knowledge should be transferred, thus the performance of our model also be improved. To show the effectiveness of our model and the importance of considering multi-level dependency relationship, we conduct experiments on several public datasets, on which we obtain significant improvements over current methods. Multi-task learning aims to train a single model on multiple related tasks jointly, so that useful knowledge learned from one task can be transferred to enhance the generalization performance of other tasks. Over the last few years, different types of multi-task learning mechanisms (; ; ;) have been proposed and proved better than single-task learning methods from natural language processing and computer vision to chemical study . Despite the success of multi-task learning, when applying to'discrete' data (graph/text), most of the current multi-task learning frameworks only leverage the general task dependency with the assumption that the task dependency remains the same for different data samples; and different sub-structures (node/word) in one data sample (graph/text). However, this assumption is not always true in many real-world problems. Different data samples may have different task dependency. For example, when we want to predict the chemical properties of a particular toxic molecule, despite the general task dependency, its representations learned from toxicity prediction tasks should be more significant than the other tasks. Even for the same data sample, different sub-structures may have different task dependency. Take sentence classification as an example. Words like'good' or'bad' may transfer more knowledge from sentiment analysis tasks, while words like'because' or'so' may transfer more from discourse relation identification tasks. In this work, to accurately learn the task dependency in both general level and data-specific level, we propose a novel framework,'Learning to Transfer via ModellIng mulTi-level Task dEpeNdency' (L2T-MITTEN). The general task dependency is learned as a parameterized weighted dependency graph. And the data-specific task dependency is learned with the position-wise mutual attention mechanism. The two-level task dependency can be used by our framework to improve the performance on multiple tasks. And the objective function of multi-task learning can further enhance the quality of the learned task dependency. By iteratively mutual enhancement, our framework can not only perform better on multiple tasks, but also can extract high-quality dependency structures at different levels, which can reveal some hidden knowledge of the datasets. Another problem is that to transfer task-specific representations between every task pair, the number of transfer functions will grow quadratically as the number of tasks increases, which is unaffordable. To solve this, we develop a universal representation space where all task-specific representations get mapped to and all target tasks can be inferred from. This decomposition method reduces the space complexity from quadratic to linear. We validate our multi-task learning framework extensively on different tasks, including graph classication, node classification, and text classification. Our framework outperforms all the other state-ofthe-art (SOTA) multi-task methods. Besides, we show that L2T-MITTEN can be used as an analytic tool to extract interpretable task dependency structures at different levels on real-world datasets. Our contributions in this work are threefold: • We propose a novel multi-task learning framework to learn to both general task dependency and data-specific task dependency. The learned task dependency structures can be mutually enhanced with the objective function of multi-task learning. • We develop a decomposition method to reduce the space complexity needed by transfer functions from quadratic to linear. • We conduct extensive experiments on different real-world datasets to show the effectiveness of our framework and the importance of modelling multi-level task dependency. According to a recent survey , existing multi-task learning methods can be categorized by whether they share the parameters hardly or softly. For hard parameter sharing, a bottom network will be shared among all the tasks, and each individual task will have its own task-specific output network. The parameter sharing of the bottom network reduces the parameter needed to be learned and thus can avoid over-fitting to a specific task. However, when the tasks are not relevant enough , the shared-bottom layers will suffer from optimization conflicts caused by mutually contradicted tasks. If the bottom model is not capable enough to encode all the necessary knowledge from different tasks, this method will fail to correctly capture all the tasks. points out that the gradients of some dominant task will be relatively larger than gradients of other tasks. This dominant phenomenon will be more obvious when the proportions of labeled data between tasks are uneven, in which case the model will be majorly optimized on data-rich tasks. To alleviate this problem, some recent works try to dynamically adjust the task weight during the training stage. casts the multi-task learning to a multi-objective optimization problem, and they use a gradient-based optimization method to find a Pareto optimal solution. proposes a new normalization method on gradients, which attempts to balance the influences of different tasks. proposes to apply Mixture-of-Experts on multi-task learning, which linearly combines different experts (bottoms) by learnable gates. Because different experts can capture different knowledge among tasks, this model can, to some extent, model the dependency relationship among tasks. Methods using soft parameter sharing (; ; ;) do not keep the shared bottom layers. Instead, for soft-parameter models, most of the model parameters are task-specific. focuses on reducing the annotation effort of the dependency parser tree. By combining two networks with a L2 normalization mechanism, knowledge from a different source language can be used to reduce the requirement of the amount of annotation. Further, in some existing works, the shallow layers of the model will be separated from other layers, and be used as the feature encoders to extract task-specific representations. For example, proposes a Cross-Stitch model which is a typical separate bottom model. Cross-Stitch model will be trained on different tasks separately to encode the task-specific representations from different bottom layers. Then, a cross-stitch unit is used a as a gate to combine those separately trained layers. introduces the tensor factorization model to allow common knowledge to be shared at each layer in the network. By the strategy proposed in their work, parameters are softly shared across the corresponding layers of the deep learning network and the parameter sharing ratio will be determined by the model itself. We also note that some recent works (; ;) can learn to capture task dependency. computes an affinity matrix among tasks based on whether the solution for one task can be sufficiently easily read out of the representation trained for another task. However, it can only capture the general task dependency. uses a sigmoid gated interaction module between two tasks to model their relation. But it will suffer from quadratic growth in space as the number of tasks increases. utilizes a shared network to share features across different tasks and uses the attention mechanism to automatically determine the importance of the shared features for the respective task. However, there is no knowledge transferred or interaction between tasks. In this section, we propose our framework L2T-MITTEN, which can end-to-end learn the task dependency in both general and data-specific level, and help to assemble multi-task representations from different task-specific encoders. To formulate our framework, we first start by briefly introducing a general setting of multi-task learning. For each task t ∈ {1, ..., T}, we have a corresponding dataset {(X k=1 with N (t) data samples, where k represent the feature vector of k-th data sample and y k is its label. We would like to train T models for these tasks, and each model has its own parameter W (t). Note that for different multi-task learning frameworks, these parameters can be shared hardly or softly. The goal of multi-task learning is to improve general performance by sharing information among related tasks. The total loss of multi-task learning is calculated as: 3.2 ARCHITECTURE OVERVIEW Figure 1: The overall architecture of our multi-task learning framework (L2T-MITTEN). For each input data, we will transfer its task-specific representations among different tasks, and assemble the transferred representations via a task-specific Interaction Unit to get the final representation for each task. As is shown in Figure 1, our framework consists of three components: Task-specific Encoders, Transfer Block, and Readout Block. The Task-specific Encoders consists of T separate feature encoders, which can be any type of feed-forward networks based on specific data. Unlike hard parameter sharing methods that tie the bottom encoders' parameters together, we keep each feature encoder separate to efficiently extract task-specific knowledge. In this way, for a given data sample X k, we can use these encoders to get task-specific representations, where E t (X k) is the representation of X k for task t. To conduct multi-task learning, one can simply use the representation of each task alone to predict labels without sharing any parameters. However, this model will suffer for tasks without sufficient labeled data. Therefore, we would like to transfer the knowledge among these T tasks and assemble the transferred representations, which is what we do in the Transfer Block. The Readout Block also consists of T separate readout modules depending on the specific data. The detailed architecture for different tasks can be found in Appendix B. In the Transfer Block, the first step is to transfer the task-specific representations from source to target tasks. A naive way is to use a transfer function F i→j (·) to transfer the task-specific representation from the space of task i to task j for every task pair: where W i→j ∈ R d×d, and d is the dimension of the task-specific representation. However, this will in a total number of T 2 transfer functions. Thus, to prevent the quadratic growth, we develop a universal representation space where all task-specific representations get mapped to and all target tasks can be inferred from. More specifically, we decompose each transfer function F i→j (·) to F T j • F Si (·). Assume that we are trying to transfer the task-specific representation E i (X k) from task i to task j, where i, j ∈ {1, 2, ..., T}. We can decompose the transfer matrix W i→j to S i and T j. In this way, we only need 2T transfer functions in total for F Si (·) and F T j (·). The space complexity is reduced from O(T 2) to O(T). Here we denote the transferred representation from task i to task j as: where S i, T j ∈ R d×d and d is a hyper-parameter. Figure 2: Position-wise mutual attention mechanism. With the transferred representations, the next step of the Transfer Block is to assemble the transferred representations with respect to the multi-level task dependency. Here, the multi-level task dependency consists of two parts: the general task dependency and the data-specific task dependency. The multi-level task dependency is modelled by the position-wise mutual attention mechanism as shown in Figure 2. To model the general task dependency, we represent it by a parameterized weighted dependency graph D ∈ R T ×T. The learnable weight of this parameterized dependency graph represents the transferable weight between any task pair. Note that the dependency graph is asymmetrical. In this way, the negative influence of irrelevant tasks can be reduced as much as possible. Further, even for the same task pair, the transferable weight (dependency) may be different for different data samples; different sub-structure (node/word) in one data sample (graph/text). Therefore, we study the data-specific task dependency in depth. To efficiently model the dataspecific task dependency, we consider the mutual attention between representations of the same data sample under source and target tasks. Given H i→j, the transferred representation from the task i to task j, and E j (X k), the original representation from the target task, we get the position-wise mutual attention by: where W Qi, W Kj ∈ R d×d are the query and key projection matrices, d is a hyper-parameter, ⊗ is the Hadamard product, and SUM is used to eliminate the last dimension (d). We use Hadamard product instead of matrix multiplication because we only want the sub-structure in a given data sample to interact with its counterpart under other tasks. Take graph data as an example, a certain node of one graph will only give attention to the same node of that graph under other tasks. Then, for a target task j, we obtain a set of general task dependency; and a set of data-specific task dependency. To integrate them, we first scale data-specific task dependency A j by the general task dependency D j. And then, we calculate the weighted sum of the transferred representations according to the multi-level task dependency. The final assembled representationX j k (for data sample k and task j) is as follow: whereD is the normalized version of D, W V i ∈ R d×d is the value projection matrix, W Oj ∈ R d×d is the output projection matrix. Note that η here is a scalar parameter used to prevent the vanish gradient problem of two normalization operations. In this section, we evaluate the performance of our proposed L2T-MITTEN approach against several classical and SOTA approaches on two application domains: graph and text. In graph domain, we train a multitask Graph Convolutional Network (GCN) for both graph-level and node-level classification. And in text domain, we train a multitask Recurrent Neural Network (RNN) for text classification. Further, we provide visualization and analysis on the learned hidden dependency structure. Codes and datasets will be released. For graph-level classification, we use Tox21 and SIDER . Tox21: Toxicology in the 21st Century (Tox21) is a database measuring the toxicity of chemical compounds. This dataset contains qualitative toxicity assays for 8014 organic molecules on 12 different targets including nuclear receptors and stress response pathways. In our experiment, we treat each molecule as a graph and each toxicity assay as a binary graph-level classification task (for 12 tasks in total). The Side Effect Resource (SIDER) is a database of marketed drugs and adverse drug reactions. This dataset contains qualitative drug side-effects measurements for 1427 drugs on 27 side-effects. In our experiment, we treat each drug (organic molecule) as a graph and the problem of predicting whether a given drug induces a side effect as a individual graph-level classification tasks (for 27 tasks in total). For node-level classification, we use DBLP and BlogCatalog . In the dataset, authors are represented by nodes and its feature is generated by titles of their papers. Two authors are linked together if they have co-authored at least two papers in 2014-2019. We use 18 representative conferences as labels. An author is assigned to multiple labels if he/she has published papers in some conferences. The processed DBLP dataset is also published in our repository. BlogCatalog: The BlogCatalog is a collection of bloggers. In the dataset, bloggers are represented as nodes, and there is a link between two bloggers if they are friends. The interests of each blogger can be tagged according to the categories that he/she published blogs in. This dataset uses 39 categories as labels. Each blogger is assigned to multiple labels if he/she published a blog in some categories. For text classification, we use TMDb 1 dataset. TMDb: The Movie Database (TMDb) dataset is a collection of information for 4803 movies. For each movie, the dataset includes information ranging from the production company, production country, release date to plot, genre, popularity, etc. In our experiment, we select plots as the input with genres as the label. We treat the problem of predicting whether a given plot belongs to a genre as an individual text-level classification tasks (for 20 tasks in total). A summary of the five datasets is provided in Appendix A. We compare our L2T-MITTEN approach with both classical and SOTA approaches. The details are given as follows: Single-task method: Single-Task: Simply train a network consists of encoder block and readout block for each task separately. Classical multi-task method: Shared-Bottom : This is a widely adopted multi-task learning framework which consists of a shared-bottom network (encoder block in our case) shared by all tasks, and a separate tower network (readout block in our case) for each specific task. The input is fed into the sharedbottom network, and the tower networks are built upon the output of the shared-bottom. Each tower will then produce the task-specific outputs. Cross-Stitch : This method uses a "cross-stitch" unit to learn the combination of shared and task-specific representation. The "cross-stitch" unit is a k × k trainable matrix (k is the number of tasks) which will transfer and fuse the representation among tasks by the following equation: where x i is the output of the lower level layer for task i, α ij is the transfer weight from task j to task i, andx i is the input of the higher level layer for task i. MMoE : This method adopts the Multi-gate Mixture-of-Expert structure. This structure consists of multiple bottom networks (experts), and multiple gating networks which take the input features and output softmax gates assembling the experts with different weights. The assembled features are then passed into the task-specific tower networks. All the baseline models use the same encoder and readout block for each task. The architecture details are provided in Appendix B. We partition the datasets into 80:20 training/testing sets (i.e. each data sample can either appear in the training or testing set) and evaluate our approach under multiple settings 2: Sufficient setting: all tasks have sufficient labeled training data; Imbalanced setting: some tasks have more labeled training data than others; Deficient setting: all tasks have deficient labeled training data. Models are trained for 100 epochs using the ADAM optimizer. We report the performance of our approach and baselines on graph classification, node classification and text classification tasks in terms of AUC-ROC score in Table 1 and 2 respectively. 3 From the above , first of all, we can see that the multi-task methods outperform the single-task method in most cases which shows the effectiveness of knowledge transfer and multi-task learning. Further, we can see that our proposed L2T-MITTEN approach outperforms both classical and SOTA in most tasks. Finally, our approach shows significant improvement under deficient labeled training data setting, since our approach leverages the structure of the data sample itself to guide the transfer among tasks. Secondly, we found that in the real-world dataset, like DBLP dataset, our model can outperform other SOTA methods significantly, which demonstrate the importance of taking multi-level dependency into consideration. Note that the Single-Task can achieve the second-best . This fact indicates that in the real-world dataset, tasks may be irrelevant to each other. Our multi-level task dependency can be more effective to prevent the influence of other irrelevant tasks. Furthermore, we conduct experiments on the text classification dataset, TMDb. The Cross-Stitch model achieves the best when the label ratio for every task is 80%. However, our task can achieve the best for partially labeled setting (partially 10%) and few labeled setting (all 10%). This fact demonstrates that our directed task dependency graph can effectively prevent the negative knowledge be transferred among different tasks when the training label is few. For visualization and analysis of the learned multi-level task dependency structure, we will take DBLP as an example here due to its simplicity in interpreting and understanding. First, in Figure 3a, where we directly visualize the learned general task dependency matrix, we can see our approach indeed captures the task dependency structure in general, i.e. conferences from the same domain are more likely to be in the same sub-tree. Moreover, in Figure 3b we plot the authors (nodes) according to the learned data-specific task dependency matrix and we can see that there are some clusters formed by authors. Further, we visualize the mean value of the data-specific task dependency for each cluster, as shown in Figure 3c. We can see that different cluster does have different task dependency. This is desirable since when predicting if an author has published papers in some conferences, authors from different domains should have different transfer weight among conferences (tasks). As a summary, it is demonstrated that our approach can capture the task dependency at multiple levels according to specific data. We propose L2T-MITTEN, a novel multi-task learning framework that employs the positionwise mutual attention mechanism to learn the multi-level task dependency; transfers the taskspecific representations between tasks with linear space-efficiency; and uses the learned multilevel task dependency to guide the inference. We design three experimental settings where training data is sufficient, imbalanced or deficient, with multiple graph/text datasets. Experimental demonstrate the superiority of our method against both classical and SOTA baselines. We also show that our framework can be used as an analytical tool to extract the task dependency structures at different levels, which can reveal some hidden knowledge of tasks and of datasets A DATASET SUMMARY Figure 4, in the Encoder Block, we use several layers of graph convolutional layers followed by the layer normalization . In the Readout Block, for graph-level task, we use set-to-set as the global pooling operator to extract the graph-level representation which is later fed to a classifier; while for node-level task, we simply eliminate the global pooling layer and feed the node-level representation directly to the classifier. Figure 4: Graph convolutional networks architecture. Note that in node-level task, the Set2Set layer (global pooling) is eliminated. The text model uses long short-term memory (LSTM) architecture in their Encoder Block, and the dot-product attention in the Readout Block, as shown in Figure 5. The dot-product attention used to get the text-level representation is as follows: where O ∈ R n×d is the output of the LSTM, H n ∈ R 1×d is the hidden state for the last word, α ∈ R n×1 is attention weight for each word, andÔ ∈ R 1×d is the text-level representation (n is the number of words, d is the feature dimension for each word).

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We propose a novel multi-task learning framework which extracts multi-view dependency relationship automatically and use it to guide the knowledge transfer among different tasks.

represent the tldr for finding the corresponding scientific article

We present a new approach, SNIP, that is simple, versatile and interpretable; it prunes irrelevant connections for a given task at single-shot prior to training and is applicable to a variety of neural network models without modifications.