Question: Represent the article for finding a single-sentence TLDR

Question: Recent efforts to combine Representation Learning with Formal Methods, commonly known as the Neuro-Symbolic Methods, have given rise to a new trend of applying rich neural architectures to solve classical combinatorial optimization problems. In this paper, we propose a neural framework that can learn to solve the Circuit Satisfiability problem. Our framework is built upon two fundamental contributions: a rich embedding architecture that encodes the problem structure and an end-to-end differentiable training procedure that mimics Reinforcement Learning and trains the model directly toward solving the SAT problem. The experimental show the superior out-of-sample generalization performance of our framework compared to the recently developed NeuroSAT method. Recent advances in neural network models for discrete structures have given rise to a new field in Representation Learning known as the Neuro-Symbolic methods. Generally speaking, these methods aim at marrying the classical symbolic techniques in Formal Methods and Computer Science to Deep Learning in order to benefit both disciplines. One of the most exciting outcomes of this marriage is the emergence of neural models for learning how to solve the classical combinatorial optimization problems in Computer Science. The key observation behind many of these models is that in practice, for a given class of combinatorial problems in a specific domain, the problem instances are typically drawn from a certain (unknown) distribution. Therefore if a sufficient number of problem instances are available, then in principle, Statistical Learning should be able to extract the common structures among these instances and produce meta-algorithms (or models) that would, in theory, outperform the carefully hand-crafted algorithms. There have been two main approaches to realize this idea in practice. In the first group of methods, the general template of the solver algorithm (which is typically the greedy strategy) is directly imported from the classical heuristic search algorithm, and the Deep Learning component is only tasked to learn the optimal heuristics within this template. In combination with Reinforcement Learning, such strategy has been shown to be quite effective for various NP-complete problems -e.g. BID16. Nevertheless, the ed model is bounded by the greedy strategy, which is sub-optimal in general. The alternative is to go one step further and let Deep Learning figure out the entire solution structure from scratch. This approach is quite attractive as it allows the model not only learn the optimal (implicit) decision heuristics but also the optimal search strategies beyond the greedy strategy. However, this comes at a price: training such models can be quite challenging! To do so, a typical candidate is Reinforcement Learning (Policy Gradient, in specific), but such techniques are usually sample inefficient -e.g. BID4. As an alternative method for training, more recently BID24 have proposed using the latent representations learned for the binary classification of the Satisfiability (SAT) problem to actually produce a neural SAT solver model. Even though using such proxy for learning a SAT solver is an interesting observation and provides us with an end-to-end differentiable architecture, the model is not directly trained toward solving a SAT problem (unlike Reinforcement Learning). As we will see later in this paper, that can indeed in poor generalization and sub-optimal models. In this paper, we propose a neural Circuit-SAT solver framework that effectively belongs to the second class above; that is, it learns the entire solution structure from scratch. More importantly, to train such model, we propose a training strategy that, unlike the typical Policy Gradient, is differentiable end-toend, yet it trains the model directly toward the end goal (similar to Policy Gradient). Furthermore, our proposed training strategy enjoys an Explore-Exploit mechanism for better optimization even though it is not exactly a Reinforcement Learning approach. The other aspect of building neural models for solving combinatorial optimization problems is how the problem instance should be represented by the model. Using classical architectures like RNNs or LSTMs completely ignores the inherent structure present in the problem instances. For this very reason, there has been recently a strong push to employ structure-aware architectures such as different variations of neural graph embedding. Most neural graph embedding methodologies are based on the idea of synchronously propagating local information on an underlying (undirected) graph that represents the problem structure. The intuition behind using local information propagation for embedding comes from the fact that many original combinatorial optimization algorithms can actually be seen propagating information. In our case, since we are dealing with Boolean circuits and circuit are Directed Acyclic Graphs (DAG), we would need an embedding architecture that take into account the special architecture of DAGs (i.e. the topological order of the nodes). In particular, we note that in many DAG-structured problems (such as circuits, computational graphs, query DAGs, etc.), the information is propagated sequentially rather than synchronously, hence a justification to have sequential propagation for the embedding as well. To this end, we propose a rich embedding architecture that implements such propagation mechanism for DAGs. As we see in this paper, our proposed architecture is capable of harnessing the structural information in the input circuits. To summarize, our contributions in this work are three-fold:(a) We propose a general, rich graph embedding architecture that implements sequential propagation for DAG-structured data. (b) We adapt our proposed architecture to design a neural Circuit-SAT solver which is capable of harnessing structural signals in the input circuits to learn a SAT solver. (c) We propose a training strategy for our architecture that is end-to-end differentiable, yet similar to Reinforcement Learning techniques, it directly trains our model toward solving the SAT problem with an Explore-Exploit mechanism. The experimental show the superior performance of our framework especially in terms of generalizing to new problem domains compared to the baseline. Deep learning on graph-structured data has recently become a hot topic in the Machine Learning community under the general umbrella of Geometric Deep Learning. Based on the assumptions they make, these models typically divide into two main categories. In the first category, the graph-structured datapoints are assumed to share the same underlying graph structure (aka the domain) and only differ based on the feature values assigned to each node or edge. The methods in this category operate in both the spatial and the frequency domains; for example, , , Graph Neural Network BID23. In the second category on the other hand, each example in the training data has its own domain (graph structure). Since the domain is varying across datapoints, these other methods mostly operate in the spatial domain and typically can be seen as the generalization of the classical CNNs (e.g. MoNet BID20) or the classical RNNs (e.g. TreeLSTM BID27, ; BID25) or both (e.g.) to the graph domain. In this paper, we extend the single layer DAG-RNN model for DAG-structured data BID3; BID25 to the more general deep version with Gated Recurrent Units, where each layer processes the input DAG either in the forward or the backward direction. On the other hand, the application of Machine Learning (deep learning in specific) to logic and symbolic computation has recently emerged as a bridge between Machine Learning and the classical Computer Science. While works such as BID11; BID1 have shown the effectiveness of (recursive) neural networks in modeling symbolic expressions, others have taken one step further and tried to learn approximate algorithms to solve symbolic NP-complete problems BID16; BID4; BID30. In particular, as opposed to black box methods (e.g. BID4 ; BID30), BID16 have shown that by incorporating the underlying graph structure of a NP-hard problem, efficient search heuristics can be learned for the greedy search algorithm. Although working in the context of greedy search introduces an inductive bias that benefits the sample efficiency of the framework, the ed algorithm is still bounded by the sub-optimality of the greedy search. More recently, Selsal et al. BID24 have introduced the NeuroSAT framework -a deep learning model aiming at learning to solve the Boolean Satisfiability problem (SAT) from scratch without biasing it toward the greedy search. In particular, they have primarily approached the SAT problem as a binary classification problem and proposed a clustering-based post-processing analysis to find a SAT solution from the latent representations extracted from the learned classifier. Although, they have shown the empirical merits of their proposed framework, it is not clear why the proposed post-processing clusetring should find the SAT solution without being explicitly trained toward that goal. In this paper, we propose a deep learning framework for the Circuit-SAT problem (a more general form of the SAT problem), but in contrast to NeuroSAT, our model is directly trained toward finding SAT solutions without requiring to see them in the training sample. In this section, we formally formulate the problem of learning on DAG-structured data and propose a deep learning framework to approach the problem. It should be noted that even though this framework has been developed for DAGs, the underlying dataset can be a general graph as long as an explicit ordering for the nodes of each graph is available. This ordering is naturally induced by the topological sort algorithm in DAGs or can be imposed on general undirected graphs to yield DAGs. Let G = V G, E G denote a Directed Acyclic Graph (DAG). We assume the the set of nodes of G are ordered according to the topological sort of the DAG. For any node v ∈ V G, π G (v) represents the set of direct predecessors of v in G. Also for a given DAG G, we define the reversed DAG, G r with the same set of nodes but reversed edges. When topologically sorted, the nodes of G r appear in the reversed order of those of G. Furthermore, for a given G, let µ G: V G → R d be a d-dimensional vector function defined on the nodes of G. We refer to µ G as a DAG function -i.e. a function that is defined on a DAG. Note that the notation µ G implicitly induces the DAG structure G along with the vector function defined on the DAG. Figure 1(a) shows an example DAG function with d = 3. Finally, let G d denote the space of all possible d-dimensional functions µ G (along with their underlying graphs G). We define the parametric functional DISPLAYFORM0 The next step is to define the mathematical form of the functional F θ. In this work, we propose: DISPLAYFORM0 Intuitively, E β: G d → G q is the embedding function that maps the input d-dimensional DAG functions into a q-dimensional DAG function space. Note that the embedding function in general may transform both the underlying DAG size/structure as well as the the DAG function defined on it. In this paper, however, we assume it only transforms the DAG function and keeps the input DAG structure intact. Once the DAG is embedded into the new space, we apply the fixed pooling function P: G q → G q on the embedded DAG function to produce a (possibly) aggregated version of it. For example, if we are interested in DAG-level predictions, P can be average pooling across all nodes of the input DAG to produce a singleton DAG; whereas, in the case of node-level predictions, P is simply the Identity function. In this paper, we set P to retrieve only the sink nodes in the input DAG. Finally, the classification function C α: G q → O is applied on the aggregated DAG function to produce the final prediction output in O. In this work, we set C α to be a multi-layer neural network. The tuple θ = α, β identifies all the free parameters of the model. The (supervised) embedding of graph-based data into the traditional vector spaces has been a hot topic recently in the Machine Learning community BID18; BID25; BID27. Many of these frameworks are based on the key idea of representing each node in the input graph by a latent vector called the node state and update these latent states via an iterative (synchronous) propagation mechanism that takes the graph structure into account. Two of these methodologies that are closely related to the proposed framework in this paper are the Gated Graph Sequence Neural Networks (GGS-NN) BID18 and DAG Recurrent Neural Networks (DAG-RNN) BID25. While GGS-NNs apply multi-level Gated Recurrent Unit (GRU) like updates in an iterative propagation scheme on general (undirected) graphs, DAG-RNNs apply simple RNN logic in a one-pass, sequential propagation mechanism from the input DAG's source nodes to its sink nodes. Our proposed framework is built upon the DAG-RNN framework BID25 but it enriches this framework further by incorporating key ideas from GGS-NNs BID18, Deep RNNs BID22 and sequence-to-sequence learning BID26. Before we explain our framework, it is worth noting that assiging input feature/state vectors to each node is equivalent to defining a DAG function in our framework. For the sake of notational simplicity, for the input DAG function µ G, we define the d-dimensional node feature vector DISPLAYFORM0 Given the node feature vectors x v for an input DAG, the update rule for the state vector at each node is defined as: DISPLAYFORM1 where GRU is the standard function applied on the input vector at node v and the aggregated state of its direct predecessors which in turn is computed by the aggregator function A: 2 DISPLAYFORM2 The aggregator function is defined as a tunable deep set function BID32 with free parameters that is invariant to the permutation of its inputs. The main difference between these proposed updates rules and the ones in DAG-RNN is in DAG-RNN, we have the simple RNN logic instead of GRU, and the aggregation logic is simply (fixed) summation. By applying the update logic in equation 2 sequentially on the nodes of the input DAG processed in the topological sort order, we compute the state vector h v for all nodes of G in one pass. This would complete the one layer (forward) embedding of the input DAG function, or E β (µ G) = δ G. Note that the same way that DAG-RNNs are the generalization of RNNs on sequences to DAGs, our proposed one-layer embedding can be seen as the generalization of GRU-NNs on sequences to DAGs. Furthermore, we introduce the reversed layers (denoted by E r) that are similar to the regular forward layers except that the input DAG is processed in the reversed order. Alternatively, reversed layers can be seen as regular layers that process the reversed version of the input DAG G r; that is, DISPLAYFORM3 The main reason we have introduced reversed layers in our framework is because in the regular forward layers, the state vector for each node is only affected by the information flowing from its ancestor nodes; whereas, the information from the descendant nodes can also be highly useful for the learning task in hand. The reversed layers provide such information for the learning task. Furthermore, the introduction of reversed layers is partly motivated by the successful application of processing sequences backwards in sequence-to-sequence learning BID26. Sequences can be seen as special-case linear DAGs; as a , reversed layers can be interpreted as the generalized version of reversing sequences. The natural extension of the one-layer embedding is the stacked L-layer version where the ith layer has its own parameters β i and output DAG function dimensionality q i. Furthermore, the stacked L layers can be sequentially applied T times in the recurrent fashion to generate the final embedding: DISPLAYFORM0 DISPLAYFORM1 where β = β 1,..., β L, H is the list of the parameters and P roj H: G q L → G d is a linear projection with the projection matrix H d×q L that simply adjusts the output dimensionality of E stack so it can Figure 1: (a) A toy example input DAG function µ G, (b) a DG-DAGRNN model that processes the input in (a) using two sequential DAG embedding layers: a forward layer followed by a reverse layer. The solid red and green arrows show the flow of information within each layer while the black arrows show the feed-forward flow of information in between the layers. Also, the dotted blue arrows show the recurrent flow of information from the last embedding layer back to the first one.be fed back to E stack as the input. In our experiments, we have found that by letting T > 1, we can significantly improve the accuracy of our models without introducing more trainable parameters. In practice, we fix the value of T during training and increase it during testing to achieve better accuracy. Also note that the L stacked layers in E stack can be any permutation of regular and reversed layers. We refer to this proposed framework as Deep-Gated DAG Recursive Neural Networks or DG-DAGRNN for short. Figure 1(b) shows an example 2-layer DG-DAGRNN model with one forward layer followed by a reversed layer. The Circuit Satisfiability problem (aka Circuit-SAT) is a fundamental NP-complete problem in Computer Science. The problem is defined as follows: given a Boolean expression consists of Boolean variables, parentheses, and logical gates (specifically And ∧, Or ∨ and Not ¬), find an assignment to the variables such that it would satisfy the original expression, aka a solution. If the expression is not satisfiable, it will be labeled as UNSAT. Moreover, when represented in the circuit format, Boolean expressions can aggregate the repetitions of the same Boolean sub-expression in the expression into one node in the circuit. This is also crucial from the learning perspective as we do not want to learn two different representations for the same Boolean sub-expression. In this section, we apply the framework from the previous section to learn a Circuit-SAT solver merely from data. More formally, a Boolean circuit can be modeled as a DAG function µ G with each node representing either a Boolean variable or a logical gate. In particular, we have µ G: V G → R 4 defined as µ G (v) = One-Hot(type(v)), where type(v) ∈ {And, Or, Not, Variable}. All the source nodes in a circuit µ G have type Variable. Moreover, each circuit DAG has only one sink node (the root node of the Boolean expression).Similar to BID24, we could also approach the Circuit-SAT problem from two different angles: predicting the circuit satisfiability problem as a binary classification problem, and solving the Circuit-SAT problem directly by generating a solution if the input circuit is indeed SAT. In BID24, solving the former is the prerequisite for solving the latter. However, that is not the case in our proposed model and since we are interested to actually solve the SAT problems, we do not focus on the binary classification problem. Nevertheless, our model can be easily adapted for SAT classification, as illustrated in Appendix A. Learning to solve SAT problems (i.e.finding a satisfying assignment) is indeed a much harder problem than SAT/UNSAT classification. In the NeuroSAT framework, BID24, the authors have proposed a post-processing unsupervised procedure to decode a solution from the latent state representations of the Boolean literals. Although this approach works empirically for many SAT problems, it is not clear that it would also work for the Circuit-SAT problems. But more importantly, it is not clear why this approach should decode the SAT problems in the first place because the objective function used in BID24 does not explicitly contain any component for solving SAT problems; in fact, the decoding procedure is added as a secondary analysis after training. In other words, the model is not optimized toward actually finding SAT assignments. In contrast, in this paper, we pursue a completely different strategy for training a neural Circuit-SAT solver. In particular, using the DG-DAGRNN framework, we learn a neural functional F θ on the space of circuits µ G such that given an input circuit, it would directly generate a satisfying assignment for the circuit if it is indeed SAT. Moreover, we explicitly train F θ to generate SAT solutions without requiring to see any actual SAT assignment during training. Our proposed strategy for training F θ is reminiscent of Policy Gradient methods in Deep Reinforcement Learning BID2.The Solver Network. We start with characterizing the components of F θ. First, the embedding function E β is set to be a multi-layer recursive embedding as in equation 3 with interleaving regular forward and reversed layers making sure that the last layer is a reversed layer so that we can read off the final outputs of the embedding from the Variable nodes (i.e. the sink nodes of the reversed DAG). The classification function C α is set to be a MLP with ReLU activation function for the hidden layers and the Sigmoid activation for the output layer. The output space here encodes the soft assignment (i.e. in range) to the corresponding variable node in the input circuit. We also refer to F θ as the solver or the policy network. The Evaluator Network. Furthermore, for any given circuit µ G, we define the soft evaluation function R G as a DAG computational graph that shares the same topology G with the circuit µ G except that the And nodes are replaced by the smooth min function, the Or nodes by the smooth max function and the Not nodes by N (z) = 1 − z function, where z is the input to the Not node. The smooth min and max functions are defined as: DISPLAYFORM0 where τ ≥ 0 is the temperature. For τ = +∞, both S max and S min are the arithmetic mean functions. As τ → 0, we have S max → max and S min → min. One can also show that ∀a = (a 1, ..., a n): min(a) < S min (a) < S max (a) < max(a). More importantly, as opposed to the min and max functions, their smooth versions are fully differentiable w.r.t. all of their inputs. As its name suggests, the soft evaluation function evaluates a soft assignment (i.e. in) to the variables of the circuit. In particular, at a low enough temperature, if for a given input assignment, R G yields a value strictly greater than 0.5, then that assignment (or its hard counterpart) can be seen as a satisfying solution for the circuit. We also refer to R G as the evaluator or the reward network. Note that the evaluator network does not have any trainable parameter. Encoding logical expressions into neural networks is not new per se as there has been recently a push to enrich deep learning with symbolic computing BID14 BID31. What are new in our framework, however, are two folds: (a) each graph example in our dataset induces a different evaluation network as opposed to having one fixed network for the entire dataset, and (b) by encoding the logical operators as smooth min and max functions, we provide a more efficient framework for back-propagating the gradients and speeding up the learning as the , as we will see shortly. The Optimization. Putting the two pieces together, we define the satisfiability function DISPLAYFORM1 Intuitively, the satisfiability function uses the solver network to produce an assignment for the input circuit and then feeds the ed assignment to the evaluator network to see if it satisfies the circuit. We refer to the final output of S θ as the satisfiability value for the input circuit, which is a real number in. Having computed the satisfiability value, we define the loss function as the smooth Step function: DISPLAYFORM2 where s = S θ (µ G) and κ ≥ 1 is a constant. By minimizing the loss function in equation 7, we push the solver network to produce an assignment that yields a higher satisfiability value S θ (µ G). For satisfiable circuits this would eventually in finding a satisfiable assignment for the circuit. However, if the input circuit is UNSAT, the maximum achievable value for S θ is 0.5 as we have shown in Appendix B. In practice though, the inclusion of UNSAT circuits in the training data slows down the training process mainly because the UNSAT circuits keep confusing the solver network as it tries hard to find a SAT solution for them. For this very reason, in this scheme, we only train on SAT examples and exclude the UNSAT circuits from training. Nevertheless, if the model has enough capacity, one can still include the UNSAT examples and pursue the training as a pure unsupervised learning task since the true SAT/UNSAT labels are not used anywhere in equation 7.Moreover, the loss function in equation 7 has a nice property of having higher gradients for satisfiability values close to 0.5 when κ > 1 (we set κ = 10 in our experiments). This means that the gradient vector in backpropagation is always dominated by the examples closer to the decision boundary. In practice, that would mean that the training algorithm immediately in the beginning pushes the easier examples in the training set (with satisfiability values close to 0.5) to the SAT region (> 0.5) with a safety margin from 0.5. As the training progresses, harder examples (with satisfiability values close to 0) start moving toward the SAT region. As mentioned before, the proposed learning scheme in this section can be seen as a variant of Policy Gradient methods, where the solver network represents the policy function and the evaluator network acts as the reward function. The main difference here is that in our problem the mathematical form of the reward function is fully known and is differentiable so the entire pipeline can be trained using backpropagation in an end-to-end fashion to maximize the total reward over the training sample. Exploration vs. Exploitation. The reason we use the smooth min and max functions in the evaluator network instead of the actual min and max is that in a min-max circuit, the gradient vector of the output of the circuit w.r.t. its inputs has at most one non-zero entry 1. That is, the circuit output is sensitive to only one of its inputs in the case of infinitesimal changes. For fixed input values, we refer to this input as the active input and to the path from the active input to the output as the active path. In the case of a min-max evaluator, the gradients flow back only through the active path of the evaluator network forcing the solver network to change such that it can satisfy the input circuit through its active path only. This strategy however is quite myopic and, as we observed empirically, leads to slow training and sub-optimal solutions. To avoid this effect, we use the smooth min and max functions in the evaluator network to allow the gradients to flow through all paths in the input circuit. Furthermore, in the beginning of the training we start with a high temperature value to let the model explore all paths in the input circuits for finding a SAT solution. As the training progresses, we slowly anneal the temperature toward 0 so that the model exploits more active path(s) for finding a solution. One annealing strategy is to let τ = t −, where t is timestep and is the annealing rate. In our experiments we set = 0.4. It should be noted that at the test time, the smooth min and max functions are replaced by their non-smooth versions. Prediction. Given a test circuit µ G, we evaluate s = S θ (µ G) = R G F θ (µ G). If s > 0.5 then the circuit is classified as SAT and the SAT solution is provided by F θ (µ G). Otherwise, the circuit is classified as UNSAT. This way, unlike SAT classification, we predict SAT for a given circuit only if we have already found a SAT solution for it. In other words, our model never produces false positives. We have formally proved this in Appendix B. Moreover, at the prediction time, we do not need to set the number of recurrences T in equation 3 to the same value we used for training. In fact, we have observed by letting T to be variable on the per example basis, we can improve the model accuracy quite significantly at the test time. The baseline method we have compared our framework to is the NeuroSAT model by BID24. Like most classical SAT solvers, NeuroSAT assumes the input problem is given in the Conjunctive Normal Form (CNF). Even though that is a fair assumption in general, in some cases, the input does not naturally come as CNF. For instance, in hardware verification, the input problems are often in the form of circuits. One can indeed convert the circuit format into CNF in polynomial time using Tseitin transformation. However, such transformation will introduce extra variables (i.e. the derived variables) which may further complicate the problem for the SAT solver. More importantly, as a number of works in the SAT community have shown, such transformations typically lose the structural information embedded in the circuit format, which otherwise can be a rich source of information for the SAT solver, BID28;;; BID12;. As a , there has been quite an effort in the classical SAT community to develop SAT solvers that directly work with the circuit format BID28; BID15. In the similar vein, our neural framework for learning a SAT solver enables us to harness such structural signals in learning by directly consuming the circuit format. That contrasts the NeuroSAT approach which cannot in principle benefit from such structural information. Despite this clear advantage of our framework to NeuroSAT, in this paper, we assume the (raw) input problems come in CNF, just so we can make a fair comparison to NeuroSAT. Instead for our method, we propose to use pre-processing methods to convert the input CNF into circuit that has the potential of injecting structural information into the circuit structure. In particular, if available, one can in principle encode problem-specific heuristics into the structure while building the circuit. For example, if there is a variable ordering heuristic available for a specific class of SAT problems, it can be used to build that target circuit in a certain way, as discussed in Appendix C. Note that we could just consume the original CNF; after all, CNF is a (flat) circuit, too. But as we empirically observed, that would negatively affect the , which again highlights the fact that our proposed framework has been optimized to utilize circuit structure as much as possible. Both our method and NeuroSAT require a large training sample size for moderate size problems. The good news is both methods can effectively be trained on an infinite stream of randomly generated problems in real-world applications. However, since we ran our experiments only on one GPU with limited memory, we had to limit the training sample size for the purpose of experimentation. This in turn restricts the maximum problem sizes we could train both models on. Nevertheless, our method can generalize pretty well to out-of-sample SAT problems with much larger sizes, as shown below. We have used the generation process proposed in the NeuroSAT paper BID24 to generate random k-SAT CNF pairs (with k stochastically set according to the default settings in BID24). These pairs are then directly fed to NeuroSAT for training. For our method, on the other hand, we first need to convert these CNFs into circuits 2. In Appendix C, we have described the details of this conversion process. Experimental Setup: We have trained a DG-DAGRNN model (i.e. our framework) and a NeuroSAT model on a dataset of 300K SAT and UNSAT pairs generated according to the scheme proposed in BID24. The number of Boolean variables in the problems in this dataset ranges from 3 to 10. We have designed both models to have roughly ∼ 180K tunable parameters. In particular our model has two DAG embedding layers: a forward layer followed by a reversed layer, each with the embedding dimension q = 100. The classifier is a 2-layer MLP with hidden dimensionality 30. The aggregator function A(·) consists of two 2-layer MLPs, each with hidden dimensionality 50. For training, we have used the Adam optimization algorithm with learning rate of 10 −5, weight decay of 10 −10 and gradient clipping norm of 0.65. We have also applied a dropout mechanism for the aggregator function during training with the rate of 20%. For the NeuroSAT model, we have used the default hyper-parameter settings proposed in BID24. Finally, since our model does not produce false positives, we have only included satisfiable examples in the test data for all experiments. In-Sample Results: Once we trained the two models, the main performance metric we measure is the percentage of SAT problems in the test set that each model can actually find a SAT solution for.3 FIG1 (Left) shows this metric on a test set from the same distribution for both our model and NeuroSAT as the number of recurrences (or propagation iterations for NeuroSAT) T increases. Not surprisingly, both methods are able to decode more SAT problems as we increase T. However, our method converges much faster than NeuroSAT (to a slightly smaller value). In other words, compared to NeuroSAT, our method requires smaller of number of iterations at the test time to decode SAT problems. We conjecture this is due to the fact the sequential propagation mechanism in DG-DAGRNN is more effective in decoding the structural information in the circuit format for the SAT problem than the synchronous propagation mechanism in NeuroSAT for the flat CNF. Out-of-Sample Results: Furthermore, we evaluated both trained models on test datasets drawn from different distributions than the training data with much larger number of variables (20, 40, 60 and 80 variables, in particular). We let both models iteratively run on each test dataset until the test metric converges. FIG1 (Right) shows the test metric for both methods on these datasets after convergence. As the demonstrate, compared to that of our method, the performance of NeuroSAT declines faster as we increase the number variables during test time, with a significant margin. In other words, our method generalizes better to out-of-sample, larger problems during the test time. We attribute this to the fact that NeuroSAT is trained toward the SAT classification problem as a proxy to learn a solver. This may in the classifier picking up certain features that are informative for classification of in-sample examples which are, otherwise, harmful (or useless at best) for learning a solver for out-of-sample examples. Our framework, on the other hand, simply does not suffer from such problem because it is directly trained toward solving the SAT problem. Time Complexity: We trained both our model and NeuroSAT for a day on a single GPU. To give an idea of the test time complexity, it took both our model and NeuroSAT roughly about 3s to run for 40 iterations on a single example of 20 variables. We also measured the time that it takes for a modern SAT Solver (MiniSAT here) to solve a similar example to be roughly about 0.7s in average. Despite this difference, our neural approach is way more prallelizable compared to modern solvers such that many examples can be solved concurrently in a single batch on GPU. For example, while it took MiniSAT 114min to solve a set of 10, 000 examples, it took our method only 8min to solve for the same set in a batch-processing fashion on GPU. This indeed shows another important advantage of our neural approach toward SAT solving in large-scale applications: the extreme parallelization. To further evaluate the generalization performance of the trained models from the previous section, we have tested them on SAT problems coming from an entire different domain than k-SAT problems. In particular, we have chosen the graph k-coloring decision problem which belongs to the class of NP-complete problems. In short, given an undirected graph G with k color values, in graph k-coloring decision problem, we seek to find a mapping from the graph nodes to the color set such that no adjacent nodes in the graph have the same color. This classical problem is reducible to SAT. Moreover, the graph topology in general contains valuable information that can be further injected into the circuit structure when preparing circuit representation for our model. Appendix D illustrates how we incorporate this information to convert instances of the graph k-coloring problem into circuits. For this experiment, we have generated two different test datasets:Dataset-1: We have generated a diverse set of random graphs with number of nodes ranging between 6 and 10 and the edge percentage of 37%. The random graphs are evenly generated according to six distinct distributions: Erdos-Renyi, Barabasi-Albert, Power Law, Random Regular, Watts-Strogatz and Newman-Watts-Strogatz. Each generated graph is then paired with a random color number in 2 ≤ k ≤ 4 to generate a graph k-coloring instance. We only keep the SAT instances in the dataset. We first generate random trees with the same number of nodes as Dataset-1. Then each tree is paired with a random color number in 2 ≤ k ≤ 4. Since the chromatic number of trees is 2, every single pair so far is SAT. Lastly, for each pair we keep adding random edges to the graph until it becomes UNSAT, then we remove the last added edge to make the instance SAT again and stop. Even though Dataset-1 has much higher coverage in terms of different graph distributions, Dataset-2 contains harder SAT examples in general, simply because in average, it contains maximally constrained instances that are still SAT. We evaluated both our method and NeuroSAT (which were both trained on k-SAT-3-10) on these test datasets. Our method could solve 48% and 27% of the SAT problems in Dataset-1 and Dataset-2, respectively. However, to our surprise, the same NeuroSAT model that generated the out-of-sample on k-SAT datasets in FIG1, could not solve any of the SAT graph k-coloring problems in Dataset-1 and Dataset-2, even after 128 propagation iterations. This does not match the reported in BID24 on graph coloring. We suspect different CNF formulations for the graph k-coloring problem might be the cause behind this discrepancy, which would mean that NeuroSAT is quite sensitive to the change of problem distribution. Nevertheless, the final judgment remains open up to further investigations. In a separate effort, we tried to actually train a fresh NeuroSAT model on a larger versions of Dataset-1 and Dataset-2 which also included UNSAT examples. However, despite a significant decrease on the classification training loss, NeuroSAT failed to decode any of the SAT problems in the test sets. We attribute this behavior to the fact that NeuroSAT is dependent on learning a good SAT classifier that can capture the conceptual essence of SAT vs. UNSAT. As a , in order to avoid learning superficial classification features, NeuroSAT restricts its training to a strict regime of SAT-UNSAT pairs, where the two examples in a pair only differ in negation of one literal. However, such strict regime can be only enforced in the random k-SAT problems. For graph coloring, the closest strategy we could come up with was the one in Dataset-2, where the SAT-UNSAT examples in a pair only differ in an edge (which still translates to a couple of clauses in the CNF). This again signifies the importance of learning the solver directly rather than relying on a classification proxy. In this paper, we proposed a neural framework for efficiently learning a Circuit-SAT solver. Our methodology relies on two fundamental contributions: a rich DAG-embedding architecture that implements the sequential propagation mechanism on DAG-structured data and is capable of learning useful representations for the input circuits, and an efficient training procedure that trains the DAGembedding architecture directly toward solving the SAT problem without requiring SAT/UNSAT labels in general. Our proposed training strategy is fully differentiable end-to-end and at the same time enjoys many features of Reinforcement Learning such as an Explore-Exploit mechanism and direct training toward the end goal. As our experiments showed, the proposed embedding architecture is able to harness structural information in the input DAG distribution and as a solve the test SAT cases in a fewer number of iterations compared to the baseline. This would also allow us to inject domain-specific heuristics into the circuit structure of the input data to obtain better models for that specific domain. Moreover, our direct training procedure as opposed to the indirect, classification-based method in NeuroSAT enables our model to generalize better to out-of-sample test cases, as demonstrated by the experiments. This superior generalization got even more expressed as we transferred the trained models to a complete new domain (i.e. graph coloring). Furthermore, we argued that not only does direct training give us superior out-of-sample generalization, but it is also essential for the problem domains where we cannot enforce the strict training regime where SAT and UNSAT cases come in pairs with almost identical structures, as proposed by BID24.Future efforts in this direction would include closely examining the SAT solver algorithm learned by our framework to see if any high-level knowledge and insight can be extracted to further aide the classical SAT solvers. Needless to say, this type of neural models have a long way to go in order to compete with industrial SAT solvers; nevertheless, these preliminary are promising enough to motivate the community to pursue this direction. We would like to thank Leonardo de Moura and Nikolaj Bjorner from Microsoft Research for the valuable feedback and discussions. In the classification problem, we are interested to merely classify each input circuit as SAT or UNSAT.To do so, we customize DG-DAGRNN framework as follows. The classification function C α is set to be a MLP with ReLU activation function for the hidden layers and the Sigmoid activation for the output layer. As the the output space O will become. The embedding function E β is set to be a multi-layer recursive embedding as in equation 3 with interleaving regular forward and reversed layers. For the classification problem, we make sure the last layer of the embedding is a forward layer so that we can read off from only one sink node (i.e. the expression root node) and feed the to the classification function for the final prediction. Finally given a labeled training set, we minimize the standard cross-entropy loss via the end-to-end backpropagation through the entire network.(Only-If) Proof by contradiction: let us assume that there exists a soft assignment a ∈ n such that R G (a) > 0.5, then using Lemma 1 and the definition of H(·), we will have:µ G H(a) = H R G (a) = I R G (a) > 0.5 = 1In other words, we have found a hard assignment H(a) that satisfies the circuit µ G; this is a contradiction. APPENDIX C: CONVERTING CNF TO CIRCUIT There are many ways one can convert a CNF to a circuit; some are optimized toward extracting structural information -e.g. BID12. Here, we have taken a more intuitive and general approach based on the Cube and Conquer paradigm (Heule et al. FORMULA1) for solving CNF-SAT problems. In the Cube and Conquer paradigm, for a given input Boolean formula F, a variable x in F is picked and set to TRUE once to obtain F x contains x, we effectively reduce the complexity of the original SAT problem by removing one variable. This process can be repeated recursively (up to a fixed level) for F + x and F − x by picking a new variable to reduce the complexity even further. Now inspired by this paradigm, one can easily show that the following logical equivalence holds for any variable x in F: DISPLAYFORM0 And this is exactly the principle we used to convert a CNF formula F into a circuit. In particular, by applying the equivalence in equation 8 recursively, up to a fixed level 4, we perform the CNF to circuit conversion (Note that F We know from Computer Science theory that the graph k-coloring problem can be reduced to the SAT problem by representing the problem as a Boolean CNF. There are many ways in the literature to do so; we have picked the Muldirect approach from . In particular, for a graph with N nodes and maximum k allowed colors, we define the Boolean variables x ij for 1 ≤ i ≤ N and 1 ≤ j ≤ k, where x ij = 1 indicates that the ith node is colored by the jth color. Then, the CNF encoding the decision graph k-coloring problem is defined as: DISPLAYFORM0 where E is the set of the graph edges. The left set of clauses in equation 9 ensure that each node of the graph takes at least one color. The right set of clauses in equation 9 enforce the constraint that the neighboring nodes cannot take the same color. As a , any satisfiable solution to the CNF in equation 9 corresponds to at least one coloring solution for the original problem if not more. Note that in this formulation, we do not require each node to take only one color value; therefore, one SAT solution can produce multiple valid graph coloring solutions. To generate a circuit from the above CNF, we note that the graph structure in graph coloring problem contains valuable structural information that can be potentially encoded as heuristics into the circuit structure. One such good heuristics, in particular, is the node degrees. More specifically, the most constrained variable first heuristic in Constraint Satisfaction Problems (CSPs) recommends assigning values to the most constrained variable first. In graph coloring problem, the higher the node degree, the more constrained the variables associated with that node are. Therefore, sorting the graph nodes based on their degrees would give us a meaningful variable ordering, which can be further used to build the circuit using the equivalence in equation 8, for example.

represent the short summary for finding the corresponding scientific article (long):

Examples:

"We detect statistical interactions captured by a feedforward multilayer neural network by directly interpreting its learned weights." == "Interpreting neural networks is a crucial and challenging task in machine learning. In this paper, we develop a novel framework for detecting statistical interactions captured by a feedforward multilayer neural network by directly interpreting its learned weights. Depending on the desired interactions, our method can achieve significantly better or similar interaction detection performance compared to the state-of-the-art without searching an exponential solution space of possible interactions. We obtain this accuracy and efficiency by observing that interactions between input features are created by the non-additive effect of nonlinear activation functions, and that interacting paths are encoded in weight matrices. We demonstrate the performance of our method and the importance of discovered interactions via experimental on both synthetic datasets and real-world application datasets. Despite their strong predictive power, neural networks have traditionally been treated as "black box" models, preventing their adoption in many application domains. It has been noted that complex machine learning models can learn unintended patterns from data, raising significant risks to stakeholders BID43. Therefore, in applications where machine learning models are intended for making critical decisions, such as healthcare or finance, it is paramount to understand how they make predictions BID6 BID17.Existing approaches to interpreting neural networks can be summarized into two types. One type is direct interpretation, which focuses on 1) explaining individual feature importance, for example by computing input gradients BID37 BID34 BID40 or by decomposing predictions BID2 BID36, 2) developing attention-based models, which illustrate where neural networks focus during inference BID20 BID27 BID47, and 3) providing model-specific visualizations, such as feature map and gate activation visualizations BID48 BID21. The other type is indirect interpretation, for example post-hoc interpretations of feature importance BID32 and knowledge distillation to simpler interpretable models BID7.It has been commonly believed that one major advantage of neural networks is their capability of modeling complex statistical interactions between features for automatic feature learning. Statistical interactions capture important information on where features often have joint effects with other features on predicting an outcome. The discovery of interactions is especially useful for scientific discoveries and hypothesis validation. For example, physicists may be interested in understanding what joint factors provide evidence for new elementary particles; doctors may want to know what interactions are accounted for in risk prediction models, to compare against known interactions from existing medical literature. In this paper, we propose an accurate and efficient framework, called Neural Interaction Detection (NID), which detects statistical interactions of any order or form captured by a feedforward neural network, by examining its weight matrices. Our approach is efficient because it avoids searching over an exponential solution space of interaction candidates by making an approximation of hidden unit importance at the first hidden layer via all weights above and doing a 2D traversal of the input weight matrix. We provide theoretical justifications on why interactions between features are created at hidden units and why our hidden unit importance approximation satisfies bounds on hidden unit gradients. Top-K true interactions are determined from interaction rankings by using a special form of generalized additive model, which accounts for interactions of variable order BID46 BID25. Experimental on simulated datasets and real-world datasets demonstrate the effectiveness of NID compared to the state-of-the-art methods in detecting statistical interactions. The rest of the paper is organized as follows: we first review related work and define notations in Section 2. In Section 3, we examine and quantify the interactions encoded in a neural network, which leads to our framework for interaction detection detailed in Section 4. Finally, we study our framework empirically and demonstrate its practical utility on real-world datasets in Section 5. Statistical interaction detection has been a well-studied topic in statistics, dating back to the 1920s when two-way ANOVA was first introduced BID11. Since then, two general approaches emerged for conducting interaction detection. One approach has been to conduct individual tests for each combination of features BID25. The other approach has been to pre-specify all interaction forms of interest, then use lasso to simultaneously select which are important BID42 BID4 BID30.Notable approaches such as ANOVA and Additive Groves BID39 belong to the first group. Two-way ANOVA has been a standard method of performing pairwise interaction detection that involves conducting hypothesis tests for each interaction candidate by checking each hypothesis with F-statistics BID45. Besides two-way ANOVA, there is also threeway ANOVA that performs the same analyses but with interactions between three variables instead of two; however, four-way ANOVA and beyond are rarely done because of how computationally expensive such tests become. Specifically, the number of interactions to test grows exponentially with interaction order. Additive Groves is another method that conducts individual tests for interactions and hence must face the same computational difficulties; however, it is special because the interactions it detects are not constrained to any functional form e.g. multiplicative interactions. The unconstrained manner by which interactions are detected is advantageous when the interactions are present in highly nonlinear data BID38. Additive Groves accomplishes this by comparing two regression trees, one that fits all interactions, and the other that has the interaction of interest forcibly removed. In interaction detection, lasso-based methods are popular in large part due to how quick they are at selecting interactions. One can construct an additive model with many different interaction terms and let lasso shrink the coefficients of unimportant terms to zero BID42. While lasso methods are fast, they require specifying all interaction terms of interest. For pairwise interaction detection, this requires O(p 2) terms (where p is the number of features), and O(2 p) terms for higherorder interaction detection. Still, the form of interactions that lasso-based methods capture is limited by which are pre-specified. Our approach to interaction detection is unlike others in that it is both fast and capable of detecting interactions of variable order without limiting their functional forms. The approach is fast because it does not conduct individual tests for each interaction to accomplish higher-order interaction detection. This property has the added benefit of avoiding a high false positive-, or false discovery rate, that commonly arises from multiple testing BID3. The interpretability of neural networks has largely been a mystery since their inception; however, many approaches have been developed in recent years to interpret neural networks in their traditional feedforward form and as deep architectures. Feedforward neural networks have undergone multiple advances in recent years, with theoretical works justifying the benefits of neural network depth (; BID23 BID33 and new research on interpreting feature importance from input gradients BID18 BID34 BID40 . Deep architectures have seen some of the greatest breakthroughs, with the widespread use of attention mechanisms in both convolutional and recurrent architectures to show where they focus on for their inferences BID20 BID27 BID47 . Methods such as feature map visualization BID48, de-convolution , saliency maps BID37, and many others have been especially important to the vision community for understanding how convolutional networks represent images. With long short-term memory networks (LSTMs), a research direction has studied multiplicative interactions in the unique gating equations of LSTMs to extract relationships between variables across a sequence BID1 BID28. DISPLAYFORM0 Figure 1: An illustration of an interaction within a fully connected feedforward neural network, where the box contains later layers in the network. The first hidden unit takes inputs from x 1 and x 3 with large weights and creates an interaction between them. The strength of the interaction is determined by both incoming weights and the outgoing paths between a hidden unit and the final output, y. Unlike previous works in interpretability, our approach extracts generalized non-additive interactions between variables from the weights of a neural network. Vectors are represented by boldface lowercase letters, such as x, w; matrices are represented by boldface capital letters, such as W. The i-th entry of a vector w is denoted by w i, and element (i, j) of a matrix W is denoted by W i,j. The i-th row and j-th column of W are denoted by W i,: and W:,j, respectively. For a vector w ∈ R n, let diag(w) be a diagonal matrix of size n × n, where {diag(w)} i,i = w i. For a matrix W, let |W| be a matrix of the same size where DISPLAYFORM0 Let [p] denote the set of integers from 1 to p. An interaction, I, is a subset of all input features [p] with |I| ≥ 2. For a vector w ∈ R p and I ⊆ [p], let w I ∈ R |I| be the vector restricted to the dimensions specified by I. 1 Consider a feedforward neural network with L hidden layers. Let p be the number of hidden units in the -th layer. We treat the input features as the 0-th layer and p 0 = p is the number of input features. There are L weight matrices DISPLAYFORM0 and L bias vectors b ∈ R p, = 1, 2,..., L. Let φ (·) be the activation function (nonlinearity), and let w y ∈ R p L and b y ∈ R be the coefficients and bias for the final output. Then, the hidden units h of the neural network and the output y with input x ∈ R p can be expressed as: DISPLAYFORM1 We can construct a directed acyclic graph G = (V, E) based on non-zero weights, where we create vertices for input features and hidden units in the neural network and edges based on the non-zero entries in the weight matrices. See Appendix A for a formal definition. A statistical interaction describes a situation in which the joint influence of multiple variables on an output variable is not additive BID8 BID39 DISPLAYFORM0 For example, in x 1 x 2 +sin (x 2 + x 3 + x 4), there is a pairwise interaction {1, 2} and a 3-way higherorder interaction {2, 3, 4}, where higher-order denotes |I| ≥ 3. Note that from the definition of statistical interaction, a d-way interaction can only exist if all its corresponding (d − 1)-interactions exist BID39. For example, the interaction {1, 2, 3} can only exist if interactions {1, 2}, {1, 3}, and {2, 3} also exist. We will often use this property in this paper. In feedforward neural networks, statistical interactions between features, or feature interactions for brevity, are created at hidden units with nonlinear activation functions, and the influences of the interactions are propagated layer-by-layer to the final output (see Figure 1). In this section, we propose a framework to identify and quantify interactions at a hidden unit for efficient interaction detection, then the interactions are combined across hidden units in Section 4. In feedforward neural networks, any interacting features must follow strongly weighted connections to a common hidden unit before the final output. That is, in the corresponding directed graph, interacting features will share at least one common descendant. The key observation is that nonoverlapping paths in the network are aggregated via weighted summation at the final output without creating any interactions between features. The statement is rigorized in the following proposition and a proof is provided in Appendix A. The reverse of this statement, that a common descendant will create an interaction among input features, holds true in most cases. Proposition 2 (Interactions at Common Hidden Units). Consider a feedforward neural network with input feature DISPLAYFORM0, there exists a vertex v I in the associated directed graph such that I is a subset of the ancestors of v I at the input layer (i.e., = 0).In general, the weights in a neural network are nonzero, in which case Proposition 2 blindly infers that all features are interacting. For example, in a neural network with just a single hidden layer, any hidden unit in the network can imply up to 2 Wj,: 0 potential interactions, where W j,: 0 is the number of nonzero values in the weight vector W j,: for the j-th hidden unit. Managing the large solution space of interactions based on nonzero weights requires us to characterize the relative importance of interactions, so we must mathematically define the concept of interaction strength. In addition, we limit the search complexity of our task by only quantifying interactions created at the first hidden layer, which is important for fast interaction detection and sufficient for high detection performance based on empirical evaluation (see evaluation in Section 5.2 and TAB3).Consider a hidden unit in the first layer: φ w x + b, where w is the associated weight vector and x is the input vector. While having the weight w i of each feature i, the correct way of summarizing feature weights for defining interaction strength is not trivial. For an interaction I ⊆ [p], we propose to use an average of the relevant feature weights w I as the surrogate for the interaction strength: µ (|w I |), where µ (·) is the averaging function for an interaction that represents the interaction strength due to feature weights. We provide guidance on how µ should be defined by first considering representative averaging functions from the generalized mean family: maximum value, root mean square, arithmetic mean, geometric mean, harmonic mean, and minimum value BID5. These options can be narrowed down by accounting for intuitive properties of interaction strength: 1) interaction strength is evaluated as zero whenever an interaction does not exist (one of the features has zero weight); 2) interaction strength does not decrease with any increase in magnitude of feature weights; 3) interaction strength is less sensitive to changes in large feature weights. While the first two properties place natural constraints on interaction strength behavior, the third property is subtle in its intuition. Consider the scaling between the magnitudes of multiple feature weights, where one weight has much higher magnitude than the others. In the worst case, there is one large weight in magnitude while the rest are near zero. If the large weight grows in magnitude, then interaction strength may not change significantly, but if instead the smaller weights grow at the same rate, then interaction strength should strictly increase. As a , maximum value, root mean square, and arithmetic mean should be ruled out because they do not satisfy either property 1 or 3. Our definition of interaction strength at individual hidden units is not complete without considering their outgoing paths, because an outgoing path of zero weight cannot contribute an interaction to the final output. To propose a way of quantifying the influence of an outgoing path on the final output, we draw inspiration from Garson's algorithm BID14 BID15, which instead of computing the influence of a hidden unit, computes the influence of features on the output. This is achieved by cumulative matrix multiplications of the absolute values of weight matrices. In the following, we propose our definition of hidden unit influence, then prove that this definition upper bounds the gradient magnitude of the hidden unit with its activation function. To represent the influence of a hidden unit i at the -th hidden layer, we define the aggregated weight z DISPLAYFORM0 where z ∈ R p. This definition upper bounds the gradient magnitudes of hidden units because it computes Lipschitz constants for the corresponding units. Gradients have been commonly used as variable importance measures in neural networks, especially input gradients which compute directions normal to decision boundaries BID34 BID16 BID37. Thus, an upper bound on the gradient magnitude approximates how important the variable can be. A full proof is shown in Appendix C. Lemma 3 (Neural Network Lipschitz Estimation). Let the activation function φ (·) be a 1-Lipschitz function. Then the output y is z DISPLAYFORM1 We now combine our definitions from Sections 3.1 and 3.2 to obtain the interaction strength ω i (I) of a potential interaction I at the i-th unit in the first hidden layer h DISPLAYFORM0.(Note that ω i (I) is defined on a single hidden unit, and it is agnostic to scaling ambiguity within a ReLU based neural network. In Section 4, we discuss our scheme of aggregating strengths across hidden units, so we can compare interactions of different orders. In this section, we propose our feature interaction detection algorithm NID, which can extract interactions of all orders without individually testing for each of them. Our methodology for interaction detection is comprised of three main steps: 1) train a feedforward network with regularization, 2) interpret learned weights to obtain a ranking of interaction candidates, and 3) determine a cutoff for the top-K interactions. Data often contains both statistical interactions and main effects BID44. Main effects describe the univariate influences of variables on an outcome variable. We study two architectures: MLP and MLP-M. MLP is a standard multilayer perceptron, and MLP-M is an MLP with additional univariate networks summed at the output (Figure 2). The univariate networks are intended to discourage the modeling of main effects away from the standard MLP, which can create spurious interactions using the main effects. When training the neural networks, we apply sparsity regularization on the MLP portions of the architectures to 1) suppress unimportant interacting paths and 2) push the modeling of main effects towards any univariate networks. We note that this approach can also generalize beyond sparsity regularization (Appendix G). Input: input-to-first hidden layer weights W, aggregated weights z DISPLAYFORM0 Output: ranked list of interaction candidates DISPLAYFORM1 1: d ← initialize an empty dictionary mapping interaction candidate to interaction strength 2: for each row w of W indexed by r do 3:for j = 2 to p do I ← sorted indices of top j weights in w 5: DISPLAYFORM0 DISPLAYFORM1Figure 2: Standard feedforward neural network for interaction detection, with optional univariate networksWe design a greedy algorithm that generates a ranking of interaction candidates by only considering, at each hidden unit, the top-ranked interactions of every order, where 2 ≤ |I| ≤ p, thereby drastically reducing the search space of potential interactions while still considering all orders. The greedy algorithm (Algorithm 1) traverses the learned input weight matrix W across hidden units and selects only top-ranked interaction candidates per hidden unit based on their interaction strengths (Equation 1). By selecting the top-ranked interactions of every order and summing their respective strengths across hidden units, we obtain final interaction strengths, allowing variable-order interaction candidates to be ranked relative to each other. For this algorithm, we set the averaging function µ (·) = min (·) based on its performance in experimental evaluation (Section 5.1).With the averaging function set to min (·), Algorithm 1's greedy strategy automatically improves the ranking of a higher-order interaction over its redundant subsets 2 (for redundancy, see Definition 1). This allows the higher-order interaction to have a better chance of ranking above any false positives and being captured in the cutoff stage. We justify this improvement by proving Theorem 4 under a mild assumption. Theorem 4 (Improving the ranking of higher-order interactions). Let R be the set of interactions proposed by Algorithm 1 with µ (·) = min (·), let I ∈ R be a d-way interaction where d ≥ 3, and let S be the set of subset (d − 1)-way interactions of I where |S| = d. Assume that for any hidden unit j which proposed s ∈ S ∩ R, I will also be proposed at the same hidden unit, and DISPLAYFORM2. Then, one of the following must be true: a) ∃s ∈ S ∩ R ranked lower than I, i.e., ω(I) > ω(s), or b) ∃s ∈ S where s / ∈ R.The full proof is included in Appendix D. Under the noted assumption, the theorem in part a) shows that a d-way interaction will improve over one its d − 1 subsets in rankings as long as there is no sudden drop from the weight of the (d − 1)-way to the d-way interaction at the same hidden units. We note that the improvement extends to b) as well, when d = |S ∩ R| > 1.Lastly, we note that Algorithm 1 assumes there are at least as many first-layer hidden units as there are the true number of non-redundant interactions. In practice, we use an arbitrarily large number of first-layer hidden units because true interactions are initially unknown. In order to predict the true top-K interactions DISPLAYFORM0, we must find a cutoff point on our interaction ranking from Section 4.2. We obtain this cutoff by constructing a Generalized Additive Model (GAM) with interactions: DISPLAYFORM1 x 1 x 2 + 2 x3+x5+x6 + 2 x3+x4+x5+x7 + sin(x 7 sin(x 8 + x 9)) + arccos(0.9x 10) DISPLAYFORM2 where g i (·) captures the main effects, g i (·) captures the interactions, and both g i and g i are small feedforward networks trained jointly via backpropagation. We refer to this model as MLP-Cutoff.We gradually add top-ranked interactions to the GAM, increasing K, until GAM performance on a validation set plateaus. The exact plateau point can be found by early stopping or other heuristic means, and we report DISPLAYFORM3 as the identified feature interactions. A variant to our interaction ranking algorithm tests for all pairwise interactions. Pairwise interaction detection has been a standard problem in the interaction detection literature BID25 BID9 due to its simplicity. Modeling pairwise interactions is also the de facto objective of many successful machine learning algorithms such as factorization machines BID31 and hierarchical lasso BID4.We rank all pairs of features {i, j} according to their interaction strengths ω({i, j}) calculated on the first hidden layer, where again the averaging function is min (·), and ω({i, j}) = p1 s=1 ω s ({i, j}). The higher the rank, the more likely the interaction exists. In this section, we discuss our experiments on both simulated and real-world datasets to study the performance of our approach on interaction detection. Averaging Function Our proposed NID framework relies on the selection of an averaging function (Sections 3.1, 4.2, and 4.4). We experimentally determined the averaging function by comparing representative functions from the generalized mean family BID5: maximum, root mean square, arithmetic mean, geometric mean, harmonic mean, and minimum, intuitions behind which were discussed in Section 3.1. To make the comparison, we used a test suite of 10 synthetic functions, which consist of a variety of interactions of varying order and overlap, as shown in TAB2. We trained 10 trials of MLP and MLP-M on each of the synthetic functions, obtained interaction rankings with our proposed greedy ranking algorithm (Algorithm 1), and counted the total number of correct interactions ranked before any false positive. In this evaluation, we ignore predicted interactions that are subsets of true higher-order interactions because the subset interactions are redundant (Section 2). As seen in FIG0, the number of true top interactions we recover is highest with the averaging function, minimum, which we will use in all of our experiments. A simple analytical study on a bivariate hidden unit also suggests that the minimum is closely correlated with interaction strength (Appendix B). TAB2 ) over 10 trials. x-axis labels are maximum, root mean square, arithmetic mean, geometric mean, harmonic mean, and minimum. Neural Network Configuration We trained feedforward networks of MLP and MLP-M architectures to obtain interaction rankings, and we trained MLP-Cutoff to find cutoffs on the rankings. In our experiments, all networks that model feature interactions consisted of four hidden layers with first-to-last layer sizes of: 140, 100, 60, and 20 units. In contrast, all individual univariate networks had three hidden layers with sizes of: 10, 10, and 10 units. All networks used ReLU activation and were trained using backpropagation. In the cases of MLP-M and MLP-Cutoff, summed networks were trained jointly. The objective functions were meansquared error for regression and cross-entropy for classification tasks. On the synthetic test suite, MLP and MLP-M were trained with L1 constants in the range of 5e-6 to 5e-4, based on parameter tuning on a validation set. On real-world datasets, L1 was fixed at 5e-5. MLP-Cutoff used a fixed L2 constant of 1e-4 in all experiments involving cutoff. Early stopping was used to prevent overfitting. Datasets We study our interaction detection framework on both simulated and real-world experiments. For simulated experiments, we used a test suite of synthetic functions, as shown in TAB2. The test functions were designed to have a mixture of pairwise and higher-order interactions, with varying order, strength, nonlinearity, and overlap. F 1 is a commonly used function in interaction detection literature BID19 BID39 BID25. All features were uniformly distributed between −1 and 1 except in F 1, where we used the same variable ranges as reported in literature BID19. In all synthetic experiments, we used random train/valid/test splits of 1/3 each on 30k data points. We use four real-world datasets, of which two are regression datasets, and the other two are binary classification datasets. The datasets are a mixture of common prediction tasks in the cal housing and bike sharing datasets, a scientific discovery task in the higgs boson dataset, and an example of very-high order interaction detection in the letter dataset. Specifically, the cal housing dataset is a regression dataset with 21k data points for predicting California housing prices BID29. The bike sharing dataset contains 17k data points of weather and seasonal information to predict the hourly count of rental bikes in a bikeshare system BID10. The higgs boson dataset has 800k data points for classifying whether a particle environment originates from the decay of a Higgs Boson BID0. Lastly, the letter recognition dataset contains 20k data points of transformed features for binary classification of letters on a pixel display BID12. For all real-world data, we used random train/valid/test splits of 80/10/10.Baselines We compare the performance of NID to that of three baseline interaction detection methods. Two-Way ANOVA utilizes linear models to conduct significance tests on the existence of interaction terms. Hierarchical lasso (HierLasso) BID4 applies lasso feature selection to extract pairwise interactions. RuleFit BID13 contains a statistic to measure pairwise interaction strength using partial dependence functions. Additive Groves (AG) BID39 ) is a nonparameteric means of testing for interactions by placing structural constraints on an additive model of regression trees. AG is a reference method for interaction detection because it directly detects interactions based on their non-additive definition. As discussed in Section 4, our framework NID can be used for pairwise interaction detection. To evaluate this approach, we used datasets generated by synthetic functions F 1 -F 10 (TAB2 that contain a mixture of pairwise and higher-order interactions, where in the case of higher-order interactions we tested for their pairwise subsets as in BID39 ; BID25 . AUC scores of interaction strength proposed by baseline methods and NID for both MLP and MLP-M are shown in TAB3 . We ran ten trials of AG and NID on each dataset and removed two trials with highest and lowest AUC scores. When comparing the AUCs of NID applied to MLP and MLP-M, we observe that the scores of MLP-M tend to be comparable or better, except the AUC for F 6 . On one hand, MLP-M performed better on F 2 and F 4 because these functions contain main effects that MLP would model as spurious interactions with other variables. On the other hand, MLP-M performed worse on F 6 because it modeled spurious main effects in the {8, 9, 10} interaction. Specifically, {8, 9, 10} can be approximated as independent parabolas for each variable (shown in Appendix I). In our analyses of NID, we mostly focus on MLP-M because handling main effects is widely considered an important problem in interaction detection BID4 BID24 BID22. Comparing the AUCs of AG and NID for MLP-M, the scores tend to close, except for F 5, F 6, and F 8, where NID performs significantly better than AG. This performance difference may be due to limitations on the model capacity of AG, which is tree-based. In comparison to ANOVA, HierLasso and RuleFit, NID-MLP-M generally performs on par or better. This is expected for ANOVA and HierLasso because they are based on quadratic models, which can have difficulty approximating the interaction nonlinearities present in the test suite. In Figure 4, heat maps of synthetic functions show the relative strengths of all possible pairwise interactions as interpreted from MLP-M, and ground truth is indicated by red cross-marks. The interaction strengths shown are normally high at the cross-marks. An exception is F 6, where NID proposes weak or negligible interaction strengths at the cross-marks corresponding to the {8, 9, 10} interaction, which is consistent with previous remarks about this interaction. Besides F 6, F 7 also shows erroneous interaction strengths; however, comparative detection performance by the baselines is similarly poor. Interaction strengths are also visualized on real-world datasets via heat maps (FIG1). For example, in the cal housing dataset, there is a high-strength interaction between x 1 and x 2. These variables mean longitude and latitude respectively, and it is clear to see that the outcome variable, California housing price, should indeed strongly depend on geographical location. We further observe high-strength interactions appearing in the heat maps of the bike sharing, higgs boson dataset, and letter datasets. For example, all feature pairs appear to be interacting in the letter dataset. The binary classification task from the letter dataset is to distinguish letters A-M from N-Z using 16 pixel display features. Since the decision boundary between A-M and N-Z is not obvious, it would make sense that a neural network learns a highly interacting function to make the distinction. We use our greedy interaction ranking algorithm (Algorithm 1) to perform higher-order interaction detection without an exponential search of interaction candidates. We first visualize our higherorder interaction detection algorithm on synthetic and real-world datasets, then we show how the predictive capability of detected interactions closes the performance gap between MLP-Cutoff and MLP-M. Next, we discuss our experiments comparing NID and AG with added noise, and lastly we verify that our algorithm obtains significant improvements in runtime.x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 TAB2 x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 DISPLAYFORM0 x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 DISPLAYFORM1 Figure 4: Heat maps of pairwise interaction strengths proposed by our NID framework on MLP-M for datasets generated by functions F 1 -F 10 (We visualize higher-order interaction detection on synthetic and real-world datasets in Figures 6 and 7 respectively. The plots correspond to higher-order interaction detection as the ranking cutoff is applied (Section 4.3). The interaction rankings generated by NID for MLP-M are shown on the x-axes, and the blue bars correspond to the validation performance of MLP-Cutoff as interactions are added. For example, the plot for cal housing shows that adding the first interaction significantly reduces RMSE. We keep adding interactions into the model until reaching a cutoff point. In our experiments, we use a cutoff heuristic where interactions are no longer added after MLP-Cutoff's validation performance reaches or surpasses MLP-M's validation performance (represented by horizontal dashed lines).As seen with the red cross-marks, our method finds true interactions in the synthetic data of F 1 -F 10 before the cutoff point. Challenges with detecting interactions are again mainly associated with F 6 and F 7, which have also been difficult for baselines in the pairwise detection setting TAB3. For the cal housing dataset, we obtain the top interaction {1, 2} just like in our pairwise test (FIG1, cal housing), where now the {1, 2} interaction contributes a significant improvement in MLP-Cutoff performance. Similarly, from the letter dataset we obtain a 16-way interaction, which is consistent with its highly interacting pairwise heat map (FIG1 . For the bike sharing and higgs boson datasets, we note that even when considering many interactions, MLP-Cutoff eventually reaches the cutoff point with a relatively small number of superset interactions. This is because many subset interactions become redundant when their corresponding supersets are found. In our evaluation of interaction detection on real-world data, we study detected interactions via their predictive performance. By comparing the test performance of MLP-Cutoff and MLP-M with respect to MLP-Cutoff without any interactions (MLP-Cutoff Ø), we can compute the relative test performance improvement obtained by including detected interactions. These relative performance improvements are shown in TAB7 for the real-world datasets as well as four selected synthetic datasets, where performance is averaged over ten trials per dataset. The of this study show DISPLAYFORM2 Figure 6: MLP-Cutoff error with added top-ranked interactions (along x-axis) of F 1 -F 10 TAB2, where the interaction rankings were generated by the NID framework applied to MLP-M. Red crossmarks indicate ground truth interactions, and Ø denotes MLP-Cutoff without any interactions. Subset interactions become redundant when their true superset interactions are found. TAB2, where the interaction rankings were generated by the NID framework on MLP-M. Ø denotes MLP-Cutoff without any interactions.that a relatively small number of interactions of variable order are highly predictive of their corresponding datasets, as true interactions should. We further study higher-order interaction detection of our NID framework by comparing it to AG in both interaction ranking quality and runtime. To assess ranking quality, we design a metric, toprank recall, which computes a recall of proposed interaction rankings by only considering those interactions that are correctly ranked before any false positive. The number of top correctly-ranked interactions is then divided by the true number of interactions. Because subset interactions are redundant in the presence of corresponding superset interactions, only such superset interactions can count as true interactions, and our metric ignores any subset interactions in the ranking. We compute the top-rank recall of NID on MLP and MLP-M, the scores of which are averaged across all tests in the test suite of synthetic functions TAB2 with 10 trials per test function. For each test, we remove two trials with max and min recall. We conduct the same tests using the stateof-the-art interaction detection method AG, except with only one trial per test because AG is very computationally expensive to run. In FIG4, we show top-rank recall of NID and AG at different Gaussian noise levels 3, and in FIG4, we show runtime comparisons on real-world and synthetic datasets. As shown, NID can obtain similar top-rank recall as AG while running orders of magnitude times faster. In higher-order interaction detection, our NID framework can have difficulty detecting interactions from functions with interlinked interacting variables. For example, a clique x 1 x 2 +x 1 x 3 +x 2 x 3 only TAB2, (b) comparison of runtimes, where NID runtime with and without cutoff are both measured. NID detects interactions with top-rank recall close to the state-of-the-art AG while running orders of magnitude times faster.contains pairwise interactions. When detecting pairwise interactions (Section 5.2), NID often obtains an AUC of 1. However, in higher-order interaction detection, the interlinked pairwise interactions are often confused for single higher-order interactions. This issue could mean that our higherorder interaction detection algorithm fails to separate interlinked pairwise interactions encoded in a neural network, or the network approximates interlinked low-order interactions as higher-order interactions. Another limitation of our framework is that it sometimes detects spurious interactions or misses interactions as a of correlations between features; however, correlations are known to cause such problems for any interaction detection method BID39 BID25. We presented our NID framework, which detects statistical interactions by interpreting the learned weights of a feedforward neural network. The framework has the practical utility of accurately detecting general types of interactions without searching an exponential solution space of interaction candidates. Our core insight was that interactions between features must be modeled at common hidden units, and our framework decoded the weights according to this insight. In future work, we plan to detect feature interactions by accounting for common units in intermediate hidden layers of feedforward networks. We would also like to use the perspective of interaction detection to interpret weights in other deep neural architectures. A PROOF AND DISCUSSION FOR PROPOSITION 2Given a trained feedforward neural network as defined in Section 2.3, we can construct a directed acyclic graph G = (V, E) based on non-zero weights as follows. We create a vertex for each input feature and hidden unit in the neural network: V = {v,i |∀i,}, where v,i is the vertex corresponding to the i-th hidden unit in the -th layer. Note that the final output y is not included. We create edges based on the non-zero entries in the weight matrices, i.e., DISPLAYFORM0 Note that under the graph representation, the value of any hidden unit is a function of parent hidden units. In the following proposition, we will use vertices and hidden units interchangeably. Proposition 2 (Interactions at Common Hidden Units). Consider a feedforward neural network with input feature DISPLAYFORM1, there exists a vertex v I in the associated directed graph such that I is a subset of the ancestors of v I at the input layer (i.e., = 0).Proof. We prove Proposition 2 by contradiction. Let I be an interaction where there is no vertex in the associated graph which satisfies the condition. Then, for any vertex v L,i at the L-th layer, the value f i of the corresponding hidden unit is a function of its ancestors at the input layer I i where I ⊂ I i.Next, we group the hidden units at the L-th layer into non-overlapping subsets by the first missing feature with respect to the interaction I. That is, for element i in I, we create an index set S i ∈ [p L]: DISPLAYFORM2 Note that the final output of the network is a weighed summation over the hidden units at the L-th layer: DISPLAYFORM3 Since that j∈Si w y j f j x Ij is not a function of x i, we have that ϕ (·) is a function without the interaction I, which contradicts our assumption. The reverse of this statement, that a common descendant will create an interaction among input features, holds true in most cases. The existence of counterexamples is manifested when early hidden layers capture an interaction that is negated in later layers. For example, the effects of two interactions may be directly removed in the next layer, as in the case of the following expression: max{w 1 x 1 + w 2 x 2, 0} − max{−w 1 x 1 − w 2 x 2, 0} = w 1 x 1 + w 2 x 2. Such an counterexample is legitimate; however, due to random fluctuations, it is highly unlikely in practice that the w 1 s and the w 2 s from the left hand side are exactly equal. We can provide a finer interaction strength analysis on a bivariate ReLU function: max{α 1 x 1 + α 2 x 2, 0}, where x 1, x 2 are two variables and α 1, α 2 are the weights for this simple network. We quantify the strength of the interaction between x 1 and x 2 with the cross-term coefficient of the best quadratic approximation. That is, β 0,..., β 5 = argmin βi,i=0,...,5 DISPLAYFORM0 Then for the coefficient of interaction {x 1, x 2}, β 5, we have that, DISPLAYFORM1 Note that the choice of the region (−1, 1) × (−1, 1) is arbitrary: for larger region (−c, c) × (−c, c) with c > 1, we found that |β 5 | scales with c −1. By the of Proposition B, the strength of the interaction can be well-modeled by the minimum value between |α 1 | and |α 2 |. Note that the factor before min{|α 1 |, |α 2 |} in Equation FORMULA13 Proof. For non-differentiable φ (·) such as the ReLU function, we can replace it with a series of differentiable 1-Lipschitz functions that converges to φ (·) in the limit. Therefore, without loss of generality, we assume that φ (·) is differentiable with |∂ x φ(x)| ≤ 1. We can take the partial derivative of the final output with respect to h i, the i-th unit at the -th hidden layer: DISPLAYFORM2 We can conclude the Lemma by proving the following inequality: DISPLAYFORM3 The left-hand side can be re-written as DISPLAYFORM4 The right-hand side can be re-written as DISPLAYFORM5 We can conclude by noting that |∂ x φ(x)| ≤ 1. Theorem 4 (Improving the ranking of higher-order interactions). Let R be the set of interactions proposed by Algorithm 1 with µ (·) = min (·), let I ∈ R be a d-way interaction where d ≥ 3, and let S be the set of subset (d − 1)-way interactions of I where |S| = d. Assume that for any hidden unit j which proposed s ∈ S ∩ R, I will also be proposed at the same hidden unit, and DISPLAYFORM0. Then, one of the following must be true: a) ∃s ∈ S ∩ R ranked lower than I, i.e., ω(I) > ω(s), or b) ∃s ∈ S where s / ∈ R.Proof. Suppose for the purpose of contradiction that S ⊆ R and ∀s ∈ S, ω(s) ≥ ω(I). Because DISPLAYFORM1 which is a contradiction. E ROC CURVES We evaluate our approach in a large p setting with pairwise interactions using the same synthetic function as in BID30. Specifically, we generate a dataset of n samples and p features {X (i), y (i) } using the function DISPLAYFORM2 where X (i) ∈ R p is the i th instance of the design matrix X ∈ R p×n, y (i) ∈ R is the i th instance of the response variable y ∈ R n×1, W ∈ R p×p contains the weights of pairwise interactions, β ∈ R p contains the weights of main effects, (i) is noise, and i = 1,..., n. W was generated as a sum of K rank one matrices, W = K k=1 a k a k. In this experiment, we set p = 1000, n = 1e4, and K = 5. X is normally distributed with mean 0 and variance 1, and (i) is normally distributed with mean 0 and variance 0.1. Both a k and β are sparse vectors of 2% nonzero density and are normally distributed with mean 0 and variance 1. We train MLP-M with the same hyperparameters as before (Section 5.1) but with a larger main network architecture of five hidden layers, with first-to-last layers sizes of 500, 400, 300, 200, and 100. Interactions are then extracted using the NID framework. From this experiment, we obtain a pairwise interaction strength AUC of 0.984 on 950 ground truth pairwise interactions, where the AUC is measured in the same way as those in TAB3. The corresponding ROC curve is shown in Figure 10. We compare the average performance of NID for different regularizations on MLP-M networks. Specifically, we compare interaction detection performance when an MLP-M network has L1, L2, or group lasso regularization 4. While L1 and L2 are common methods for regularizing neural network weights, group lasso is used to specifically regularize groups in the input weight matrix because weight connections into the first hidden layer are especially important in this work. In particular, we study group lasso by 1) forming groups associated with input features, and 2) forming groups associated with hidden units in the input weight matrix. In this experimental setting, group lasso effectively conducts variable selection for associated groups. BID35 define group lasso regularization for neural networks in Equation 5. Denote group lasso with input groups a R (i) GL and group lasso with hidden unit groups as R (h) GL. In order to apply both group and individual level sparsity, BID35 further define sparse group lasso in Equation 7. Denote sparse group lasso with input groups as R (i) SGL and sparse group lasso with hidden unit groups as R (h) Networks that had group lasso or sparse group lasso applied to the input weight matrix had L1 regularization applied to all other weights. In our experiments, we use large dataset sizes of 1e5 and tune the regularizers by gradually increasing their respective strengths from zero until validation performance worsens from underfitting. The rest of the neural network hyperparameters are the same as those discussed in Section 5.1. In the case of the group lasso and sparse group lasso experiments, L1 norms were tuned the same as in the standard L1 experiments. In Table 4, we report average pairwise interaction strength AUC over 10 trials of each function in our synthetic test suite TAB2 for the different regularizers. Table 4: Average AUC of pairwise interaction strengths proposed by NID for different regularizers. Evaluation was conducted on the test suite of synthetic functions TAB2. DISPLAYFORM0 average 0.94 ± 2.9e−2 0.94 ± 2.5e−2 0.95 ± 2.4e−2 0.94 ± 2.5e−2 0.93 ± 3.2e−2 0.94 ± 3.0e−2 We perform experiments with our NID approach on synthetic datasets that have binary class labels as opposed to continuous outcome variables (e.g. TAB2). In our evaluation, we compare our method against two logistic regression methods for multiplicative interaction detection, Factorization Based High-Order Interaction Model (FHIM) BID30 and Sparse High-Order Logistic Regression (Shooter). In both comparisons, we use dataset sizes of p = 200 features and n = 1e4 samples based on MLP-M's fit on the data and the performance of the baselines. We also make the following modifications to MLP-M hyperparameters based on validation performance: the main MLP-M network has first-to-last layer sizes of 100, 60, 20 hidden units, the univariate networks do not have any hidden layers, and the L1 regularization constant is set to 5e−4. All other hyperparameters are kept the same as in Section 5.1. When used in a logistic regression model, FHIM detects pairwise interactions that are predictive of binary class labels. For this comparison, we used data generated by Equation 5 in BID30, with K = 2 and sparsity factors being 5% to generate 73 ground truth pairwise interactions. In TAB8, we report average pairwise interaction detection AUC over 10 trials, with a maximum and a minimum AUC removed. FHIM NID average 0.925 ± 2.3e−3 0.973 ± 6.1e−3Shooter Min et al. FORMULA13 developed Shooter, an approach of using a tree-structured feature expansion to identify pairwise and higher-order multiplicative interactions in a L1 regularized logistic regression model. This approach is special because it relaxes our hierarchical hereditary assumption, which requires subset interactions to exist when their corresponding higher-order interaction also exists (Section 3). Specifically, Shooter relaxes this assumption by only requiring at least one (d − 1)-way interaction to exist when its corresponding d-way interaction exists. With this relaxed assumption, Shooter can be evaluated in depth per level of interaction order. We compare NID and Shooter under the relaxed assumption by also evaluating NID per level of interaction order, where Algorithm 1 is specifically being evaluated. We note that our method of finding a cutoff on interaction rankings (Section 4.3) strictly depends on the hierarchical hereditary assumption both within the same interaction order and across orders, so instead we set cutoffs by thresholding the interaction strengths by a low value, 1e−3.For this comparison, we generate and consider interaction orders up to degree 5 (5-way interaction) using the procedure discussed in, where the interactions do not have strict hierarchical correspondence. We do not extend beyond degree 5 because MLP-M's validation performance begins to degrade quickly on the generated dataset. The sparsity factor was set to be 5%, and to simplify the comparison, we did not add noise to the data. In TAB9, we report precision and recall scores of Shooter and NID, where the scores for NID are averaged over 10 trials. While Shooter performs near perfectly, NID obtains fair precision scores but generally low recall. When we observe the interactions identified by NID per level of interaction order, we find that the interactions across levels are always subsets or supersets of another predicted interaction. This strict hierarchical correspondence would inevitably cause NID to miss many true interactions under this experimental setting. The limitation of Shooter is that it must assume the form of interactions, which in this case is multiplicative. 87.5% 100% 90% ± 11% 21% ± 9.6% 3 96.0% 96.0% 91% ± 8.4% 19% ± 8.5% 4 100% 100% 60% ± 13% 21% ± 9.3% 5 100% 100% 73% ± 8.4% 30% ± 13% In the synthetic function F 6 TAB3, the {8, 9, 10} interaction, x 2 8 + x 2 9 + x 2 10, can be approximated as main effects for each variable x 8, x 9, and x 10 when at least one of the three variables is close to −1 or 1. Note that in our experiments, these variables were uniformly distributed between −1 and 1., and x 10. The MLP-M was trained on data generated from synthetic function F 6 (TAB3). Note that the plots are subject to different levels of bias from the MLP-M's main multivariate network. For example, let x 10 = 1 and z 2 = x 2 8 + x By symmetry under the assumed conditions, where c is a constant. In FIG10, we visualize the x 8, x 9, x 10 univariate networks of a MLP-M (Figure 2) that is trained on F 6. The plots confirm our hypothesis that the MLP-M models the {8,9,10} interaction as spurious main effects with parabolas scaled by In FIG13, we visualize the interaction between longitude and latitude for predicting relative housing price in California. This visualization is extracted from the longitude-latitude interaction network within MLP-Cutoff 5, which was trained on the cal housing dataset BID29. We can see that this visualization cannot be generated by longitude and latitude information in additive form, but rather the visualization needs special joint information from both features. The highly interacting nature between longitude and latitude confirms the high rank of this interaction in our NID experiments (see the {1, 2} interaction for cal housing in FIG1."

We propose a neural framework that can learn to solve the Circuit Satisfiability problem from (unlabeled) circuit instances.

Represent the TLDR for finding the corresponding scientific article

Are HMMs a special case of RNNs? We investigate a series of architectural transformations between HMMs and RNNs, both through theoretical derivations and empirical hybridization and provide new insights.