Question: Represent the scientific article to find a short TLDR

Question: We propose a method to incrementally learn an embedding space over the domain of network architectures, to enable the careful selection of architectures for evaluation during compressed architecture search. Given a teacher network, we search for a compressed network architecture by using Bayesian Optimization (BO) with a kernel function defined over our proposed embedding space to select architectures for evaluation. We demonstrate that our search algorithm can significantly outperform various baseline methods, such as random search and reinforcement learning . The compressed architectures found by our method are also better than the state-of-the-art manually-designed compact architecture ShuffleNet . We also demonstrate that the learned embedding space can be transferred to new settings for architecture search, such as a larger teacher network or a teacher network in a different architecture family, without any training. In many application domains, it is common practice to make use of well-known deep network architectures (e.g., VGG BID30, GoogleNet BID33, ResNet BID8) and to adapt them to a new task without optimizing the architecture for that task. While this process of transfer learning is surprisingly successful, it often in over-sized networks which have many redundant or unused parameters. Inefficient network architectures can waste computational resources and over-sized networks can prevent them from being used on embedded systems. There is a pressing need to develop algorithms that can take large networks with high accuracy as input and compress their size while maintaining similar performance. In this paper, we focus on the task of compressed architecture search -the automatic discovery of compressed network architectures based on a given large network. One significant bottleneck of compressed architecture search is the need to repeatedly evaluate different compressed network architectures, as each evaluation is extremely costly (e.g., backpropagation to learn the parameters of a single deep network can take several days on a single GPU). This means that any efficient search algorithm must be judicious when selecting architectures to evaluate. Learning a good embedding space over the domain of compressed network architectures is important because it can be used to define a distribution on the architecture space that can be used to generate a priority ordering of architectures for evaluation. To enable the careful selection of architectures for evaluation, we propose a method to incrementally learn an embedding space over the domain of network architectures. In the network compression paradigm, we are given a teacher network and we aim to search for a compressed network architecture, a student network that contains as few parameters as possible while maintaining similar performance to the teacher network. We address the task of compressed architecture search by using Bayesian Optimization (BO) with a kernel function defined over our proposed embedding space to select architectures for evaluation. As modern neural architectures can Computationally Efficient Architecture: There has been great progress in designing computationally efficient network architectures. Representative examples include SqueezeNet BID16, MobileNet , MobileNetV2 BID29, CondenseNet and ShuffleNet BID39. Different from them, we aim to develop an algorithm that can automatically search for an efficient network architecture with minimal human efforts involved in the architecture design. Neural Architecture Search (NAS): NAS has recently been an active research topic BID41 BID42 BID28 BID27 BID20 b; BID24. Some existing works in NAS are focused on searching for architectures that not only can achieve high performance but also respect some resource or computation constraints BID1 BID34 BID4 BID12 BID5. NAO BID24 and our work share the idea of mapping network architectures into a latent continuous embedding space. But NAO and our work have fundamentally different motivations, which further lead to different architecture search frameworks. NAO maps network architectures to a continuous space such that they can perform gradient based optimization to find better architectures. However, our motivation for learning the embedding space is to find a principled way to define a kernel function between architectures with complex skip connections and multiple branches. Our work is also closely related to N2N BID1, which searches for a compressed architecture based on a given teacher network using reinforcement learning. Our search algorithm is developed based on Bayesian Optimization (BO), which is different from N2N and many other existing works. We will compare our approach to other BO based NAS methods in the next paragraph. Readers can refer to BID6 for a more complete literature review of NAS.Bayesian Optimization (BO): BO is a popular method for hyper-parameter optimization in machine learning. BO has been used to tune the number of layers and the size of hidden layers BID3 BID32, the width of a network BID31 or the size of the filter bank BID2, along with other hyper-parameters, such as the learning rate, number of iterations. BID25, BID17 and BID38 also fall into this category. Our work is also related to BID9, which presents a Bayesian method for identifying the Pareto set of multi-objective optimization problems and applies the method to searching for a fast and accurate neural network. However, most existing works on BO for NAS only show on tuning network architectures where the connections between network layers are fixed, i.e., most of them do not optimize how the layers are connected to each other. BID18 proposes a distance metric OTMANN to compare network architectures with complex skip connections and branch layers, based on which NASBOT is developed, a BO based NAS framework, which can tune how the layers are connected. Although the OTMANN distance is designed with thoughtful choices, it is defined based on some empirically identified factors that can influence the performance of a network, rather than the actual performance of networks. Different from OTMANN, the distance metric (or the embedding) for network architectures in our algorithm is automatically learned according to the actual performance of network architectures instead of manually designed. Our work can also be viewed as carrying out optimization in the latent space of a high dimensional and structured space, which shares a similar idea with previous literature BID23 BID7. For example, BID23 presents a new variational auto-encoder to map kernel combinations produced by a context-free grammar into a continuous latent space. Deep Kernel Learning: Our work is also related to recent works on deep kernel learning BID36 b). They aim to learn more expressive kernels by representing the kernel function as a neural network to incorporate the expressive power of deep networks. The follow-up work extends the kernel representation to recurrent networks to model sequential data. Our work shares a similar motivation with them and tries to learn a kernel function for the neural architecture domain by leveraging the expressive power of deep networks. In this work, we focus on searching for a compressed network architecture based on a given teacher network and our goal is to find a network architecture which contains as few parameters as possible but still can obtain a similar performance to the teacher network. Formally, we aim to solve the following optimization problem: DISPLAYFORM0 where X denotes the domain of neural architectures and the function f (x): X → R evaluates how well the architecture x meets our requirement. We adopt the reward function design in N2N BID1 for the function f, which is defined based on the compression ratio of the architecture x and its validation performance after being trained on the training set. More details about the exact form of f are given in Appendix 6.1 due to space constraints. As evaluating the value of f (x) for a specific architecture x is extremely costly, the algorithm must judiciously select the architectures to evaluate. To enable the careful selection of architectures for evaluation, we propose a method to incrementally learn an embedding space over the domain of network architecture that can be used to generate a priority ordering of architectures for evaluation. In particular, we develop the search algorithm based on BO with a kernel function defined over our proposed embedding space. In the following text, we will first introduce the sketch of the BO algorithm and then explain how the proposed embedding space is used in the loop of BO.We adopt the Gaussian process (GP) based BO algorithms to maximize the function f (x), which is one of the most popular algorithms in BO. A GP prior is placed on the function f, parameterized by a mean function µ(·): X → R and a covariance function or kernel k(·, ·): X × X → R. To search for the solution, we start from an arbitrarily selected architecture x 1. At step t, we evaluate the architecture x t, i.e., obtaining the value of f (x t). Using the t evaluated architectures up to now, we compute the posterior distribution on the function f: DISPLAYFORM1 where f (x 1:t) = [f (x 1),..., f (x t)] and µ t (x) and σ 2 t (x) can be computed analytically based on the GP prior BID35. We then use the posterior distribution to decide the next architecture to evaluate. In particular, we obtain x t+1 by maximizing the expected improvement acquisition function EI t (x): X → R, i.e., x t+1 = arg max x∈X EI t (x). The expected improvement function EI t (x) BID26 measures the expected improvement over the current maximum value according to the posterior distribution: DISPLAYFORM2 Published as a conference paper at ICLR 2019 where E t indicates the expectation is taken with respect to the posterior distribution at step t p (f (x) | f (x 1:t)) and f * t is the maximum value among {f (x 1),..., f (x t)}. Once obtaining x t+1, we repeat the above described process until we reach the maximum number of steps. Finally, we return the best evaluated architecture as the solution. The main difficulty in realizing the above optimization procedure is the design of the kernel function k(·, ·) for X and the maximization of the acquisition function EI t (x) over X, since the neural architecture domain X is discrete and highly complex. To overcome these difficulties, we propose to learn an embedding space for the neural architecture domain and define the kernel function based on the learned embedding space. We also propose a search space, a subset of the neural architecture domain, over which maximizing the acquisition function is feasible and sufficient. The kernel function, which measures the similarity between network architectures, is fundamental for selecting the architectures to evaluate during the search process. As modern neural architectures can have multiple layers, multiple branches and multiple skip connections, comparing two architectures is non-trivial. Therefore, we propose to map a diverse range of discrete architectures to a continuous embedding space through the use of recurrent neural networks and then define the kernel function based on the learned embedding space. We use h(·; θ) to denote the architecture embedding function that generates an embedding for a network architecture according to its configuration parameters. θ represents the weights to be learned in the architecture embedding function. With h(·; θ), we define the kernel function k(x, x ; θ) based on the RBF kernel: DISPLAYFORM0 where σ is a hyper-parameter. h(·; θ) represents the proposed learnable embedding space and k(x, x ; θ) is the learnable kernel function. They are parameterized by the same weights θ. In the following text, we will first introduce the architecture embedding function h(·; θ) and then describe how we learn the weights θ during the search process. The architecture embedding function needs to be flexible enough to handle a diverse range of architectures that may have multiple layers, multiple branches and multiple skip connections. Therefore we adopt a Bidirectional LSTM to represent the architecture embedding function, motivated by the layer removal policy network in N2N BID1. The input to the Bi-LSTM is the configuration information of each layer in the network, including the layer type, how this layer connects to other layers, and other attributes. After passing the configuration of each layer to the Bi-LSTM, we gather all the hidden states, apply average pooling to these hidden states and then apply L 2 normalization to the pooled vector to obtain the architecture embedding. We would like to emphasize that our representation for layer configuration encodes the skip connections between layers. Skip connections have been proven effective in both human designed network architectures, such as ResNet BID8 and DenseNet BID13, and automatically found network architectures BID41. N2N only supports the kind of skip connections used in ResNet BID8 and does not generalize to more complex connections between layers, where our representation is still applicable. We give the details about our representation for layer configuration in Appendix 6.2.The weights of the Bi-LSTM θ, are learned during the search process. The weights θ determine the architecture embedding function h(·; θ) and the kernel k(·, ·; θ). Further, θ controls the GP prior and the posterior distribution of the function value conditioned on the observed data points. The posterior distribution guides the search process and is essential to the performance of our search algorithm. Our goal is to learn a θ such that the function f is consistent with the GP prior, which will in a posterior distribution that accurately characterizes the statistical structure of the function f.Let D denote the set of evaluated architectures. In step t, D = {x 1, . . ., x t}. For any architecture DISPLAYFORM1 based on the GP prior, where \ refers to the set difference operation, f (x i) is the value obtained by evaluating the architecture DISPLAYFORM2 is the posterior probability of f (x i) conditioned on the other evaluated architectures in D. The higher the value of DISPLAYFORM3 is, the more accurately the posterior distribution characterizes the statistical structure of the function f and the more the function f is consistent with the GP prior. Therefore, we learn θ by minimizing the negative log posterior probability: DISPLAYFORM4 is a Gaussian distribution and its mean and covariance matrix can be computed analytically based on k(·, ·; θ). Thus L is differentiable with respect to θ and we can learn the weights θ using backpropagation. In each optimization step, we obtain the next architecture to evaluate by maximizing the acquisition function EI t (·) over the neural architecture domain X. On one hand, maximizing EI t (·) over all the network architectures in X is unnecessary. Since our goal is to search for a compressed architecture based on the given teacher network, we only need to consider those architectures that are smaller than the teacher network. On the other hand, maximizing EI t (·) over X is non-trivial. Gradientbased optimization algorithms cannot be directly applied to optimize EI t (·) as X is discrete. Also, exhaustive exploration of the whole domain is infeasible. This calls for a search space that covers the compressed architectures of our interest and easy to explore. Motivated by N2N BID1, we propose a search space for maximizing the acquisition function, which is constrained by the teacher network, and provides a practical method to explore the search space. We define the search space based on the teacher network. The search space is constructed by all the architectures that can be obtained by manipulating the teacher network with the following three operations: layer removal, layer shrinkage and adding skip connections. Layer removal and shrinkage: The two operations ensure that we only consider architectures that are smaller than the given big network. Layer removal refers to removing one or more layers from the network. Layer shrinkage refers to shrinking the size of layers, in particular, the number of filters in convolutional layers, as we focus on convolutional networks in this work. Different from N2N, we do not consider shrinking the kernel size, padding or other configurable variables and we find that only shrinking the number of filters already yields satisfactory performance. The operation of adding skip connections is employed to increase the network complexity. N2N BID1, which uses reinforcement learning to search for compressed network architectures, does not support forming skip connections in their action space. We believe when searching for compressed architectures, adding skip connections to the compressed network is crucial for it to achieve similar performance to the teacher network and we will show ablation study to verify this. The way we define the search space naturally allows us to explore it by sampling the operations to manipulate the architecture of the teacher network. To optimize the acquisition function over the search space, we randomly sample architectures in the search space by randomly sampling the operations. We then evaluate EI t (·) over the sampled architectures and return the best one as the solution. We also have tried using evolutionary algorithm to maximize EI t (·) but it yields similar with random sampling. So for the sake of simplicity, we use random sampling to maximize EI t (·). We attribute the good performance of random sampling to the thoughtful design of the operations to manipulate the teacher network architecture. These operations already favor the compressed architectures of our interest. We implement the search algorithm with the proposed learnable kernel function but notice that the highest function value among evaluated architectures stops increasing after a few steps. We conjecture this is due to that the learned kernel is overfitted to the training samples since we only evaluate hundreds of architectures in the whole search process. An overfitted kernel may bias the following sampled architectures for evaluation. To encourage the search algorithm to explore more diverse architectures, we propose a multiple kernel strategy, motivated by the bagging algorithm, which is usually employed to avoid overfitting. In bagging, instead of training one single model on the whole dataset, multiple models are trained on different subsets of the whole dataset. Likewise, in each step of the search process, we train multiple kernel functions on uniformly sampled subsets of D, the set of all the available evaluated architectures. Technically, learning multiple kernels refers to learning multiple architecture embedding spaces, i.e., multiple sets of weights θ. After training the kernels, each kernel is used separately to compute one posterior distribution and determine one architecture to evaluate in the next step. That is to say, if we train K kernels in the current step, we will obtain K architectures to evaluate in the next step. The proposed multiple kernel strategy encourages the search process to explore more diverse architectures and can help find better architectures than training one single kernel only. When training kernels, we randomly initialize their weights and learn the weights from the scratch on subsets of evaluated architectures. We do not learn the weights of the kernel based on the weights learned in the last step, i.e., fine-tuning the Bi-LSTM from the last step. The training of the Bi-LSTM is fast since we usually only evaluate hundreds of architectures during the whole search process. A formal sketch of our search algorithm in shown Algorithm 1. We first extensively evaluate our algorithm with different teacher architectures and datasets. We then compare the automatically found compressed architectures to the state-of-the-art manually-designed compact architecture, ShuffleNet BID39. We also evaluate the transfer performance of the learned embedding space and kernel. We perform ablation study to understand how the number of kernels K and other design choices in our search algorithm influence the performance. Due to space constraints, the ablation study is included in Appendix 6.3. We use two datasets: CIFAR-10 and CIFAR-100 BID19 ). CIFAR-10 contains 60K images in 10 classes, with 6K images per class. CIFAR-100 also contains 60K images but in 100 classes, with 600 images per class. Both CIFAR-10 and CIFAR-100 are divided into a training set with 50K images and a test set with 10K images. We sample 5K images from the training set as the validation set. We provide on four architectures as the teacher network: VGG-19 BID30, ResNet-18, ResNet-34 and ShuffleNet BID39.We consider two baselines algorithms for comparison: random search (RS) and a reinforcement learning based approach, N2N BID1. Here we use RS to directly maximize the compression objective f (x). To be more specific, RS randomly samples architectures in the search space, then evaluates all of them and returns the best architecture as the optimal solution. In the following experiments, RS evaluates 160 architectures. For our proposed method, we run 20 architecture search steps, where each step generates K = 8 architectures for evaluation based on the the K different kernels. This means our proposed method evaluates 160 (20 × 8) architectures in total during the search process. Note that when evaluating an architecture during the search process, we only train it for 10 epochs to reduce computation time. So for both RS and our method, we fully train the top 4 architectures among the 160 evaluated architectures and choose the best one as the solution. When learning the kernel function parameters, we randomly sample from the set of the evaluated architectures with a probability of 0.5 to form the training set for one kernel. The of N2N are from the original paper BID1.The compression are summarized in TAB1. For a compressed network x,'Ratio' refers to the compression ratio of x, which is defined as 1 − #params(x) #params(xteacher).' Times' refers to the ratio between the size of the teacher network and the size of the compressed network, i.e., #params(xteacher) #params(x). We also show the value of f (x) as an indication of how well each architecture x meets our requirement in terms of both the accuracy and the compression ratio. For'Random Search' and'Ours', we run the experiments for three times and report the average as well as the standard deviation. We first apply our algorithm to compress three popular network architectures: VGG-19, ResNet-18 and ResNet-34, and use them as the teacher network. We can see that on both CIFAR-10 and CIFAR-100, our proposed method consistently finds architectures that can achieve higher value of f (x) than all baselines. For VGG-19 on CIFAR-100, the architecture found by our algorithm is 8 times smaller than the original teacher network while the accuracy only drops by 2.3%. For ResNet-18 on CIFAR-100, the architecture found by our algorithm has a little bit more parameters than that found by RS but has higer accuracy by about 4%. For ResNet-34 on CIFAR-100, the architecture found by our proposed method has a higher accuracy as the architecture discovered by RS but only uses about 65% of the number of parameters. Also for ResNet-34 on CIFAR-100, N2N only provides the of layer removal, denoted as'N2N -removal'.' Ours -removal' refers to only considering the layer removal operation in the search space for fair comparison. We can see that'Ours -removal' also significantly outperforms'N2N -removal' in terms of both the accuracy and the compression ratio. ShuffleNet is an extremely computation-efficient human-designed CNN architecture BID39. We also have tried to use ShuffleNet as the teacher network and see if we can further optimize this architecture. As shown in TAB1, our search algorithm successfully compresses'ShuffleNet 1 × (g = 2)' by 10.43× and 4.74× on CIFAR-10 and CIFAR-100 respectively and the compressed architectures can still achieve similar accuracy to the original teacher network. Here'1×' indicates the number of channels in the teacher ShuffleNet and'(g = 2)' indicates that the number of groups is 2. Readers can refer to BID39 for more details about the specific configuration. We now compare the compressed architectures found by our algorithm to the state-of-the-art manually-designed compact network architecture ShuffleNet. We vary the number of channels and the number of groups in ShuffleNet and compare the compressed architectures found by our proposed method against these different configurations of ShuffleNet. We conduct experiments on CIFAR-100 and the are summarized in TAB2. For'Ours' in TAB2, we use the mean of 3 runs of our method. In TAB2, VGG-19, ResNet-18, ResNet-34 and ShuffleNet refer to the compressed architectures found by our algorithm based on the corresponding teacher network and do not refer to the original architecture indicated by the name. The teacher ShuffleNet used in the experiments is'ShuffleNet 1×(g = 2)' as mentioned above.'0.5×(g = 1)' and so on in TAB2 refer to different configurations of ShuffleNet and we show the accuracy of these original ShuffleNet in the table. The compressed architectures found based on ResNet-18 and ResNet-34 have a similar number of parameters with ShuffleNet 1.5× but they can all achieve much higher accuracy than ShuffleNet 1.5×. The compressed architecture found based on ShuffleNet 1 × (g = 2) can obtain higher accuracy than ShuffleNet 0.5× while using a similar number of parameters. We now study the transferability of the learned embedding space or the learned kernel. We would like to know to what extent a kernel learned in one setting can be generalized to a new setting. To be more specific about the kernel transfer, we first learn one kernel or multiple kernels in the source setting. Then we maximize the acquisition function within the search space in the target setting and the acquisition function is computed based on the kernel learned in the source setting. The maximizer of the acquisition function is a compressed architecture for the target setting. We evaluate this architecture in the target setting and compare it with the architecture found by applying algorithms directly to the target setting. 67.71% 0.26M 1.5 × (g = 1) 72.43% 2.09M 0.5 × (g = 2)67.54% 0.27M 1.5 × (g = 2) 71.41% 2.07M 0.5 × (g = 3) 67.23% 0.27M 1.5 × (g = 3) 71.05% 2.03M 0.5 × (g = 4) 66.83% 0.27M We consider the following four settings: (a) ResNet-18 on CIFAR-10, (b) ResNet-34 on CIFAR-10, (c) VGG-19 on CIFAR-10 and (d) ResNet-18 on CIFAR-100.'ResNet-18 on CIFAR-10' refers to searching for a compressed architecture with ResNet-18 as the teacher network for the dataset CIFAR-10 and so on. We first run our search algorithm in setting (a) and transfer the learned kernel to setting (b), (c) and (d) respectively to see how much the learned kernel can transfer to a larger teacher network in the same architecture family (this means a larger search space), a different architecture family (this means a totally different search space) or a harder dataset. DISPLAYFORM0 We learn K kernels in the source setting (a) and we transfer all the K kernels to the target setting, which will in K compressed architectures for the target setting. We report the best one among the K architectures. We have tried K = 1 and K = 8 and the are shown in TAB3. In all the three transfer scenarios, the learned kernel in the source setting (a) can help find reasonably good architectures in the target setting without actually training the kernel in the target setting, whose performance is better than the architecture found by applying N2N directly to the target setting. These proves that the learned architecture embedding space or the learned kernel is able to generalize to new settings for architecture search without any additional training. We address the task of searching for a compressed network architecture by using BO. Our proposed method can find more efficient architectures than all the baselines on CIFAR-10 and CIFAR-100. Our key contribution is the proposed method to learn an embedding space over the domain of network architectures. We also demonstrate that the learned embedding space can be transferred to new settings for architecture search without any training. Possible future directions include extending our method to the general NAS problem to search for desired architectures from the scratch and combining our proposed embedding space with BID9 to identify the Pareto set of the architectures that are both small and accurate. We now discuss the form of the function f. We aim to find a network architecture which contains as few parameters as possible but still can obtain a similar performance to the teacher network. Usually compressing a network leads to the decrease in the performance. So the function f needs to provide a balance between the compression ratio and the performance. In particular, we hope the function f favors architectures of high performance but low compression ratio more than architectures of low performance but high compression ratio. So we adopt the reward function design in N2N BID1 for the function f. Formally, f is defined as: DISPLAYFORM0 where C(x) is the compression ratio of the architecture x, A(x) is the validation performance of x and A(x teacher) is the validation performance of the teacher network. The compression ratio C(x) is defined as C(x) = 1 − #params(x) #params(xteacher). Note that for any x, to evaluate f (x) we need to train the architecture x on the training data and test on the validation data. This is time-consuming so during the search process, we do not fully train x. Instead, we only train x for a few epochs and use the validation performance of the network obtained by early stopping as an approximation for A(x). We also employ the Knowledge Distillation (KD) strategy BID10 for faster training as we are given a teacher network. But when we fully train the architecture x to see its true performance, we fine tune it from the weights obtained by early stopping with cross entropy loss without using KD. We represent the configuration of one layer by a vector of length (m + 2n + 6), where m is the number of types of layers we consider and n is the maximum number of layers in the network. The first m dimensions of the vector are a one-hot vector, indicating the type of the layer. Then the following 6 numbers indicate the value of different attributes of the layer, including the kernel size, stride, padding, group, input channels and output channels of the layer. If one layer does not have any specific attribute, the value of that attribute is simply set to zero. The following 2n dimensions encode the edge information of the network, if we view the network as a directed acyclic graph with each layer as a node in the graph. In particular, the 2n dimensions are composed of two n-dim vectors, where one represents the edges incoming to the code and the other one represents the edges outgoing from the node. The nodes in the directed acyclic graph can be topologically sorted, which will give each layer an index. For an edge from node i to j, the (j − i) th element in the outgoing vector of node i and the incoming vector of node j will be 1. We are sure that j is larger than i because all the nodes are topologically sorted. With this representation, we can describe the connection information in a complex network architecture. Impact of number of kernels K: We study the impact of the number of kernels K. We conduct experiments on CIFAR-100 and use ResNet-34 as the teacher network. We vary the value of K and fix the number of evaluated architectures to 160. The are summarized in TAB4. We can see that K = 4, 8, 16 yield much better than K = 1. Also the performance is not sensitive to K as K = 4, 8, 16 yield similar . In our main experiments, we fix K = 8.Impact of adding skip connections: Our search space is defined based on three operations: layer removal, layer shrinkage and adding skip connections. A key difference between our search space and N2N BID1 is that they only support layer removal and shrinkage do not support adding skip connections. To validate the effectiveness of adding skip connections, we conduct experiments on CIFAR-100 and on three architectures. In TAB5,'Ours -removal + shrink' refers to the search space without considering adding skip connections and'Ours' refers to using the full search space. We can see that'Ours' consistently outperforms'Ours -removal + shrink' across different teacher networks, proving the effectiveness of adding skip connections. Impact of the maximization of the acquisition function: As mentioned in Section 3.2, we have two choices to maximize the acquisition function EI t (x): randomly sampling (RS) and evolutionary algorithm (EA). We conduct the experiments to compare RS and ES to compress ResNet-34 on CIFAR-100. We find that although EA is empirically better than RS in terms of maximizing EI t (x), EA is slightly worse than RS in terms of the final search performance as shown in TAB6. For any EI t (x), the solution found by EA x EA may be better than the solution found by RS x RS, i.e., EI t (x EA) > EI t (x RS). However, we observe that f (x EA) and f (x RS) are usually similar. We also plot the values of f (x) for the evaluated architectures when using RS and EA to maximize the acquisition function respectively in FIG0. We can see that the function value of the evaluated architectures grows slightly more stable when using RS to maximize the acquisition function then using EA. Therefore, we choose RS in the following experiments for the sake of simplicity.6.4 COMPARISON TO TPE (BERGSTRA ET AL., 2011)Neural architecture search can be viewed as an optimization problem in a high-dimensional and discrete space. There are existing optimization methods such as TPE BID3 and SMAC BID15 that are proposed to handle such input spaces. To further justify our idea to learn a latent embedding space for the neural architecture domain, we now compare our method to directly using TPE to search for compressed architectures in the original hyperparameter value domain. TPE BID3 ) is a hyperparameter optimization algorithm based on a tree of Parzen estimator. In TPE, they use Gaussian mixture models (GMM) to fit the probability density of the hyperparameter values, which indicates that they determine the similarity between two architecture configurations based on the Euclidean distance in the original hyperparameter value domain. However, instead of comparing architecture configurations in the original hyperparameter value domain, our method transforms architecture configurations into a learned latent embedding space and compares them in the learned embedding space. We first do not consider adding skip connections between layers and focus on layer removal and layer shrinkage only, i.e., we search for a compressed architecture by removing and shrinking layers from the given teacher network. Therefore, the hyperparameters we need to tune include for each layer whether we should keep it or not and the shrinkage ratio for each layer. This in 64 hyperparameters for ResNet-18 and 112 hyperparameters for ResNet-34. We conduct the experiments on CIFAR-100 and the are summarized in the TAB7. Comparing'TPE -removal + shrink' and'Ours -removal + shrink', we can see that our method outperforms TPE and can achieve higher accuracy with a similar size. Now, we conduct experiments with adding skip connections. Besides the hyperparameters mentioned above, for each pair of layers where the output dimension of one layer is the same as the input dimension of another layer, we tune a hyperparameter representing whether to add a skip connection between them. The in 529 and 1717 hyperparameters for ResNet-18 and ResNet-34 respectively. In this representation, the original hyperparameter space is extremely high-dimensional and we think it would be difficult to directly optimize in this space. We can see from the table that for ResNet-18, the'TPE' are worse than'TPE -removal + shrink'. We do not show the'TPE' for ResNet-34 here because the networks found by TPE have too many skip connections, which makes it very hard to train. The loss of those networks gets diverged easily and do not generate any meaningful . Based on the on'layer removal + layer shrink' only and the on the full search space, we can see that our method is better than optimizing in the original space especially when the original space is very high-dimensional. We need to randomly sample architectures in the search space when optimizing the acquisition function. As mentioned in Section 3.2, we sample the architectures by sampling the operations to manipulate the architecture of the teacher network. During the process, we need to make sure the layers in the network are still compatible with each other in terms of the dimension of the feature map. Therefore, We impose some conditions when we sample the operations in order to maintain the consistency between between layers. For layer removal, only layers whose input dimension and output dimension are the same are allowed to be removed. For layer shrinkage, we divide layers into groups and for layers in the same group, the number of channels are always shrunken with the same ratio. The layers are grouped according to their input and output dimension. For adding skip connections, only when the output dimension of one layer is the same as the input dimension of another layer, the two layers can be connected. When there are multiple incoming edges for one layer in the computation graph, the outputs of source layers are added up to form the input for that layer. When compressing ShuffleNet, we also slightly modify the original architecture before compression. We insert a 1 × 1 convolutional layer before each average pooling layer. This modification increases parameters by about 10% and does not significantly influence the performance of ShuffleNet. Note that the modification only happens when we need to compress ShuffleNet and does not influence the performance of the original ShuffleNet shown in TAB2. We observe that the random search (RS) baseline which maximizes f (x) with random sampling can achieve very good performance. To analyze RS in more detail, we show the value of f (x) for the 160 architectures evaluated in the search process in FIG1. The specific setting we choose is ResNet-34 on CIFAR-100. We can see that although RS can sometimes sample good architectures with high f (x) value, it is much more unstable than our method. The function value of the evaluated architectures selected by our method has a strong tendency to grow as we search more steps while RS does not show such trend. Also, from the histogram of values of f (x), we can see that RS has a much lower chance to get architectures with high function values than our method. This is expected since our method leverages the learned architecture embedding or the kernel function to carefully select the architecture for evaluation while RS just randomly samples from the search space. We can conclude that our method is much more efficient than RS. We discuss the possible choices of the objective function for learning the embedding space in this section. In our experiments, we learn the LSTM weights θ by maximizing the predictive posterior probability, i.e., minimizing the negative log posterior probability as defined in Eq. 5. There are two other alternative choices for the objective function as suggested by the reviewers. We discuss the two choices and compare them to our choice in the following text. Intuitively, a meaningful embedding space should be predictive of the function value, i.e, the performance of the architecture. Therefore, a reasonable choice of the objective function is to train the LSTM by regressing the function value with a Euclidean loss. Technically, this is done by adding a fully connected layer F C(·; θ) after the embedding, whose output is the predicted performance of the input architecture. However, directly training the LSTM by regressing the function value does not let us directly evaluate how accurate the posterior distribution characterizes the statistical structure of the function. As mentioned before, the posterior distribution guides the search process by influencing the choice of architectures for evaluation at each step. Therefore, we believe maximizing the predictive posterior probability is a more suitable training objective for our search algorithm than regressing the function value. To validate this, we have tried changing the objective function from Eq. 5 to the squared Euclidean distance between the predicted function value and the true function value: DISPLAYFORM0 The are summarized in TAB8. We observe that maximizing the predictive posterior probability consistently yields better than the Euclidean loss. Another possible choice of the objective function is to maximize the log marginal likelihood log p (f (D) | D; θ), which is the conventional objective function for kernel learning BID36 b). We do not choose to maximize log marginal likelihood because we empirically find that maximizing the log marginal likelihood yields worse than maximizing the predictive GP posterior as shown in TAB8. When using the log marginal likelihood, we observe that the loss is numerically unstable due to the log determinant of the covariance matrix in the log marginal likelihood. The training objective usually goes to infinity when the dimension of the covariance matrix is larger than 50, even with smaller learning rates, which may harm the search performance. Therefore, we learn the embedding space by maximizing the predictive GP posterior instead of the log marginal likelihood.

We propose a method to incrementally learn an embedding space over the domain of network architectures, to enable the careful selection of architectures for evaluation during compressed architecture search.

We present a new approach, SNIP, that is simple, versatile and interpretable; it prunes irrelevant connections for a given task at single-shot prior to training and is applicable to a variety of neural network models without modifications.


Question: represent the text to find a very short summary (1-2 sentences max)

Question: Reinforcement Learning (RL) problem can be solved in two different ways - the Value function-based approach and the policy optimization-based approach - to eventually arrive at an optimal policy for the given environment. One of the recent breakthroughs in reinforcement learning is the use of deep neural networks as function approximators to approximate the value function or q-function in a reinforcement learning scheme. This has led to with agents automatically learning how to play games like alpha-go showing better-than-human performance. Deep Q-learning networks (DQN) and Deep Deterministic Policy Gradient (DDPG) are two such methods that have shown state-of-the-art in recent times. Among the many variants of RL, an important class of problems is where the state and action spaces are continuous --- autonomous robots, autonomous vehicles, optimal control are all examples of such problems that can lend themselves naturally to reinforcement based algorithms, and have continuous state and action spaces. In this paper, we adapt and combine approaches such as DQN and DDPG in novel ways to outperform the earlier for continuous state and action space problems. We believe these are a valuable addition to the fast-growing body of on Reinforcement Learning, more so for continuous state and action space problems. Reinforcement learning (RL) is about an agent interacting with the environment, learning an optimal policy, by trail and error, for sequential decision making problems in a wide range of fields such that the agent learns to control a system and maximize a numerical performance measure that expresses a long-term objective BID6 ). Many interesting real-world control tasks, such as driving a car or riding a snowboard, require smooth continuous actions taken in response to high-dimensional, real-valued sensory input. In applying RL to continuous problems, the most common approach for a long time has been first to discretize state and action spaces and then to apply an RL algorithm for a discrete stochastic system BID2 ). However, this discretization approach has a number of drawbacks. Hence, the formulation of the reinforcement learning problem with continuous state and action space holds great value in solving more real-world problems. The advent of deep learning has had a significant impact on many areas in machine learning, dramatically improving the state-of-the-art tasks such as object detection, speech recognition and language translation BID5. The most important property of deep learning is that deep neural networks can automatically find low-dimensional representations of high-dimensional data. Therefore, deep learning enables RL to scale to problems which were previously intractable -setting with high dimensional/continuous state and action space BID0.Few of the current state of the art methods in the area of Deep RL are:• Deep Q-learning Networks (DQN) -introduced novel concepts which helped in using neural networks as function approximators for reinforcement learning algorithms (for continuous state space) BID8.• Prioritized Experience Replay (PER) -builds upon DQN with some newer approaches to outperform DQN (for continuous state space) BID10.• Deep Deterministic Policy Gradients (DDPG) -follows a different paradigm as compared to the above methods. It uses DQN as the function approximator while building on the seminal work of BID12 (for both continuous state and action space) on deterministic policy gradients. We propose a new algorithm, Prioritized DDPG using the ideas proposed in DQN, prioritized experience replay and DDPG such that it outperforms the DDPG in the continuous state and action space. Prioritized DDPG uses the concept of prioritized sampling in the function approximator of DDPG. Our show that prioritized DDPG outperforms DDPG in a majority of the continuous action space environments. We then use the concept of parameter space noise for exploration and show that this further improves the rewards achieved. Critic methods rely exclusively on function approximation and aim at learning a "good" approximation of the value functionKonda &. We survey a few of the recent best known critic methods in RL. As in any value function-based approach, DQN method tries to find the value function for each state and then tries to find the optimal function. Min et al. BID12 also consider the approach of continuous state spaces. The novelty of their approach is that they use a non-linear function approximator efficiently. Until their work, non-linear function approximators were inefficient and also had convergence issues. This was due to the fact that there was a lot of correlation between the data being fed to the neural networks which ed in them diverging (neural networks assume that the data comes from independent and identically distributed source). To overcome this drawback, the novel ideas that were proposed which made it possible to use non-linear function approximator for reinforcement learning are the following:• Experience Replay Buffer: In this technique, the input given to the neural network is selected at random from a large buffer of stored observations, which ensures that there are no correlations in the data, which is a requirement for neural networks.• Periodic Target Network Updates: The authors propose that having two sets of parameters for the same neural networks can be beneficial. The two sets of parameters are used, one of the parameters is used to compute the target at any given iteration and the other, network parameters are used in the loss computation and are updated by the first network parameters periodically, this also ensures lesser co-relation. The prioritized experience replay algorithm is a further improvement on the deep Q-learning methods and can be applied to both DQN and Double DQN. The idea proposed by the authors is as follows: instead of selecting the observations at random from the replay buffer, they can be chosen based on some criteria which will help in making the learning faster. Intuitively, what they are trying here is to replace those observations which do not contribute to learning or learning with more useful observations. To select these more useful observations, the criteria used was the error of that particular observation. This criterion helps in selecting those observations which help the agent most in learning and hence speeds up the learning process. The problem with this approach is that greedy prioritization focuses on a small subset of the experience and this lack of diversity might lead to over-fitting. Hence, the authors introduce a stochastic sampling method that interpolates between pure greedy and uniform random prioritization. Hence, the new probability if sampling a transition i is DISPLAYFORM0 where p i is the priority of transition i and α determines how much prioritization is used. This approach, while it improves the has a problem of changing the distribution of the expectation. This is resolved by the authors by using Importance Sampling (IS) weights DISPLAYFORM1 where if β = 1, the non-uniform probabilities P (i) are fully compensated BID10 ). Actor methods work with a parameterized family of policies. The gradient of the performance, with respect to the actor parameter is directly estimated by simulation, and the parameters are updated in the direction of improvement . The most popular policy gradient method is the deterministic policy gradient(DPG) method and in this approach, instead of having a stochastic policy which we have seen till now, the authors make the policy deterministic and then determine the policy gradient. The deterministic policy gradient is the expected gradient of the action-value function, which integrates over the state space; whereas in the stochastic case, the policy gradient integrates over both state and action spaces. What this leads to is that the deterministic policy gradient can be estimated more efficiently than the stochastic policy gradient. The DPG algorithm, presented by BID12 maintains a parameterized actor function µ(s|θ µ) which is the current deterministic policy that maps a state to an action. They used the normal Bellman equation to update the critic Q(s, a). They then went on to prove that the derivative expected return with respect to actor parameters is the gradient of the policy's performance. Actor critic models (ACM) are a class of RL models that separate the policy from the value approximation process by parameterizing the policy separately. The parameterization of the value function is called the critic and the parameterization of the policy is called the actor. The actor is updated based on the critic which can be done in different ways, while the critic is update based on the current policy provided by the actor BID4 BID3 ). The DDPG algorithm tries to solve the reinforcement learning problem in the continuous action and state space setting. The authors of this approach extend the idea of deterministic policy gradients. What they add to the DPG approach is the use of a non-linear function approximator BID7 ).While using a deterministic policy, the action value function reduces from DISPLAYFORM0 to DISPLAYFORM1 as the inner expectation is no longer required. What this also tells us is that the expectation depends only on the environment and nothing else. Hence, we can learn off-policy, that is, we can train our reinforcement learning agent by using the observations made by some other agent. They then use the novel concepts used in DQN to construct their function approximator. These concepts could not be applied directly to continuous action space, as there is an optimization over the action space at every step which is in-feasible when there is a continuous action space BID7 ).Once we have both the actor and the critic networks with their respective gradients, we can then use the DQN concepts -replay buffer and target networks to train these two networks. They apply the replay buffer directly without any modifications but make small changes in the way target networks are used. Instead of directly copying the values from the temporary network to the target network, they use soft updates to the target networks. The proposed algorithm is primarily an adaptation of DQN and DDPG with ideas from the work of BID10 on continuous control with deep reinforcement learning to design a RL scheme that improves on DDPG significantly. The intuition behind the idea is as follows: The DDPG algorithm uses the DQN method as a sub-algorithm and any improvement over the DQN algorithm should ideally in the improvement of the DDPG algorithm. But from the above-described methods, not all algorithms which improve DQN can be used to improve DDPG. That is because some of them need the environment to have discrete action spaces. So, for our work, we will consider only prioritized experience replay method which does not have this constraint. Now, the improvement to the DQN algorithm, the prioritized action replay method can be integrated into the DDPG algorithm in a very simple way. Instead of using just DQN as the function approximator, we can use DQN with prioritized action replay. That is, in the DDPG algorithm, instead of selecting observations randomly, we select the observations using the stochastic sampling method as defined in equation 1. The pseudo-code for the prioritized action replay is given in Algorithm 1.This algorithm is quite similar to the original DDPG algorithm with the only changes being the way the observations are selected for training in line 11 and the transition probabilities are being updated in line 16. The first change ensures we are selecting the better set of observations which help in learning faster and the second change helps in avoiding over-fitting as it ensures all the observations have a non-zero probability of being selected to train the network and only a few high error observations are not used multiple times to train the network. The proposed, prioritized DDPG algorithm was tested on many of the standard RL simulation environments that have been used in the past for bench-marking the earlier algorithms. The environments are available as part of the Mujoco platform BID13 ). Mujoco is a physics environment which was created to facilitate research and development in robotics and similar areas, where fast simulation is an important component. Initialize a random process N for action exploration 6:Receive initial observation state s 1 for t = 1, T do 8:Select action a t = µ(s t |θ µ) + N t according to the current policy 9:Execute action a t and observe reward r t and observe new state s t+1 Store transition (s t, a t, r t, s t+1) in R Storing to the replay buffer 11:Sample a mini-batch of N transitions (s i, a i, r i, s i+1) from R from R each such that - DISPLAYFORM0 Stochastic sampling 12: DISPLAYFORM1 Update critic by minimizing the loss: DISPLAYFORM2 Update the actor policy using the sampled policy gradient: DISPLAYFORM3 Update the target networks: DISPLAYFORM4 Update the transition priorities for the entire batch based on the error 17:end for 18: end for This set of environments provide a varied set of challenges for the agent as environments have continuous action as well as state space. All the environments contain stick figures with some joints trying to perform some basic task by performing actions like moving a joint in a particular direction or applying some force using one of the joints. The implementation used for making the comparison was the implementation of DDPG in baselines BID9 ). The prioritized DDPG algorithm was implemented by extending the existing code in baselines. The following are the of the prioritized DDPG agent as compared to the DDPG work . The overall reward -that is the average of the reward across all epochs until that point and reward history -average of the last 100 epochs on four environments are plotted. The yaxis represents the reward the agent has received from the environment and the x-axis is the number of epochs with each epoch corresponding to 2000 time steps. As seen in FIG0, the Prioritized DDPG algorithm reaches the reward of the DDPG algorithm in less than 300 epochs for the HalfCheetah environment. This shows that the prioritized DDPG algorithm is much faster in learning. The same trend can be observed in FIG0 for HumanoidStandup, Hopper and Ant environments. That is, prioritized DDPG agent learns and gets the same reward as DDPG much faster. This helps is in reducing overall training time. Prioritized DDPG algorithm can also help in achieving which might not be achieved by DDPG even after large number of epochs. This can be seen in the case of the Ant environment. FIG0 shows that DDPG rewards are actually declining with more training. On the other hand, Prioritized DDPG has already achieved a reward much higher and is more stable. There is no best exploration strategy in RL. For some algorithms, random exploration works better and for some greedy exploration. But whichever strategy is used, the important requirement is that the agent has explored enough about the environment and learns the best policy. BID9 in their paper explore the idea of adding noise to the agent's parameters instead of adding noise in the action space. In their paper Parameter Space Noise For Exploration, they explore and compare the effects of four different kinds of noises• Uncorrelated additive action space noise• Correlated additive Gaussian action space noise• Adaptive-param space noise• No noise at all They show that adding parameter noise vastly outperforms existing algorithms or at least is just as good on majority of the environments for DDPG as well as other popular algorithms such as Trust Region Policy Optimization BID11 ). Hence, we use the concept of parametric noise in PDDPG algorithm to improve the rewards achieved by our agent. The PDDPG algorithm with parameter noise was run on the same set of environments as the PDDPG algorithm -the Mujoco environments. The empirical are as follows. As we can see from figures 2,3 and 4 we see a great amount of variation on the reward achieved. We can infer that prioritized DDPG clearly works better for adaptive-param and corelated noise as compared to uncorrelated noise. This could be due to the fact that prioritized DDPG already explores faster as compared to DDPG and hence adding more randomness for exploration is not going to bear any fruit. Therefore, we can conclude that, PDDPG learns faster than DDPG and with the appropriate noise, it can be improved further. This can be seen in figure 5, where the overall best of both the algorithms have been plotted against each other. We see that PDDPG outperforms DDPG in majority of the environments and does reasonably well in the others. To summarize, this paper discusses the state of the art methods in reinforcement learning with our improvements that have led to RL algorithms in continuous state and action spaces that outperform the existing ones. The proposed algorithm combines the concept of prioritized action replay with deep deterministic policy gradients. As it has been shown, on a majority of the mujoco environments this algorithm vastly outperforms the DDPG algorithm both in terms of overall reward achieved and the average reward for any hundred epochs over the thousand epochs over which both were run. Hence, it can be concluded that the proposed algorithm learns much faster than the DDPG algorithm. Secondly, the fact that current reward is higher coupled with the observation that rate of increase in reward also being higher for the proposed algorithm, shows that it is unlikely for DDPG algorithm to surpass the of the proposed algorithm on that majority of environments. Also, certain kinds of noises further improve PDDPG to help attain higher rewards. One other important is that different kinds of noises work better for different environments which was evident in how drastically the changed based on the parameter noise. The presented algorithm can also be extended and improved further by finding more concepts in value based methods, which can be used in policy based methods. The overall improvements in the area of continuous space and action state space can help in making reinforcement learning more applicable in real world scenarios, as the real world systems provide continuous inputs. These methods can potentially be extended to safety critical systems, by incorporating the notion of safety during the training of a RL algorithm. This is currently a big challenge because of the necessary unrestricted exploration process of a typical RL algorithm.

Improving the performance of an RL agent in the continuous action and state space domain by using prioritised experience replay and parameter noise.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We isolate one factor of RL generalization by analyzing the case when the agent only overfits to the observations. We show that architectural implicit regularizations occur in this regime.


Question: represent the article for finding a single-sentence tldr

Question: Natural Language Inference (NLI) task requires an agent to determine the logical relationship between a natural language premise and a natural language hypothesis. We introduce Interactive Inference Network (IIN), a novel class of neural network architectures that is able to achieve high-level understanding of the sentence pair by hierarchically extracting semantic features from interaction space. We show that an interaction tensor (attention weight) contains semantic information to solve natural language inference, and a denser interaction tensor contains richer semantic information. One instance of such architecture, Densely Interactive Inference Network (DIIN), demonstrates the state-of-the-art performance on large scale NLI copora and large-scale NLI alike corpus. It's noteworthy that DIIN achieve a greater than 20% error reduction on the challenging Multi-Genre NLI (MultiNLI) dataset with respect to the strongest published system. Natural Language Inference (NLI also known as recognizing textual entiailment, or RTE) task requires one to determine whether the logical relationship between two sentences is among entailment (if the premise is true, then the hypothesis must be true), contradiction (if the premise is true, then the hypothesis must be false) and neutral (neither entailment nor contradiction). NLI is known as a fundamental and yet challenging task for natural language understanding, not only because it requires one to identify the language pattern, but also to understand certain common sense knowledge. In TAB0, three samples from MultiNLI corpus show solving the task requires one to handle the full complexity of lexical and compositional semantics. The previous work on NLI (or RTE) has extensively researched on conventional approaches BID25; BID39. Recent progress on NLI is enabled by the availability of 570k human annotated dataset and the advancement of representation learning technique. Among the core representation learning techniques, attention mechanism is broadly applied in many NLU tasks since its introduction: machine translation BID15, abstractive summarization BID50, Reading Comprehension, dialog system BID41, etc. As described by BID57, "An attention function can be described as mapping a query and a set of key-value pairs to an output, where the query, keys, values, and output are all vectors. The output is computed as a weighted sum of the values, where the weight assigned to each value is computed by a compatibility function of the query with the corresponding key". Attention mechanism is known for its alignment between representations, focusing one part of representation over another, and modeling the dependency regardless of sequence length. Observing attention's powerful capability, we hypothesize that the attention weight can assist with machine to understanding the text. A regular attention weight, the core component of the attention mechanism, encodes the crosssentence word relationship into a alignment matrix. However, a multi-head attention weightVaswani et al. can encode such interaction into multiple alignment matrices, which shows a more powerful alignment. In this work, we push the multi-head attention to a extreme by building a word- by-word dimension-wise alignment tensor which we call interaction tensor. The interaction tensor encodes the high-order alignment relationship between sentences pair. Our experiments demonstrate that by capturing the rich semantic features in the interaction tensor, we are able to solve natural language inference task well, especially in cases with paraphrase, antonyms and overlapping words. We dub the general framework as Interactive Inference Network(IIN). To the best of our knowledge, it is the first attempt to solve natural language inference task in the interaction space. We further explore one instance of Interactive Inference Network, Densely Interactive Inference Network (DIIN), which achieves new state-of-the-art performance on both SNLI and MultiNLI copora. To test the generality of the architecture, we interpret the paraphrase identification task as natural language inference task where matching as entailment, not-matching as neutral. We test the model on Quora Question Pair dataset, which contains over 400k real world question pair, and achieves new state-of-the-art performance. We introduce the related work in Section 2, and discuss the general framework of IIN along with a specific instance that enjoys state-of-the-art performance on multiple datasets in Section 3. We describe experiments and analysis in Section 4. Finally, we conclude and discuss future work in Section 5. The early exploration on NLI mainly rely on conventional methods and small scale datasets BID40. The availability of SNLI dataset with 570k human annotated sentence pairs has enabled a good deal of progress on natural language understanding. The essential representation learning techniques for NLU such as attention, memory and the use of parse structure are studied on the SNLI which serves as an important benchmark for sentence understanding. The models trained on NLI task can be divided into two categories: (i) sentence encoding-based model which aims to find vector representation for each sentence and classifies the relation by using the concatenation of two vector representation along with their absolute element-wise difference and element-wise product.(ii) Joint feature models which use the cross sentence feature or attention from one sentence to another.After neural attention mechanism is successfully applied on the machine translation task, such technique has became widely used in both natural language process and computer vision domains. Many variants of attention technique such as hard-attention, self-attention, multi-hop attention BID27, bidirectional attention BID51 and multi-head attention BID57 are also introduced to tackle more complicated tasks. Before this work, neural attention mechanism is mainly used to make alignment, focusing on specific part of the representation. In this work, we want to show that attention weight contains rich semantic information required for understanding the logical relationship between sentence pair. Though RNN or LSTM are very good for variable length sequence modeling, using Convolutional neural network in NLU tasks is very desirable because of its parallelism in computation. Convolutional structure has been successfully applied in various domain such as machine translation BID26, sentence classification BID34 ), text matching BID30 and sentiment analysis BID33, etc. The convolution structure is also applied on different level of granularity such as byte BID65, character, word BID26 and sentences levels. The Interactive Inference Network (IIN) is a hierarchical multi-stage process and consists of five components. Each of the components is compatible with different type of implementations. Potentially all exiting approaches in machine learning, such as decision tree, support vector machine and neural network approach, can be transfer to replace certain component in this architecture. We focus on neural network approaches below. 1. Embedding Layer converts each word or phrase to a vector representation and construct the representation matrix for sentences. In embedding layer, a model can map tokens to vectors with the pre-trained word representation such as GloVe BID48, word2Vec BID42 and fasttext BID32. It can also utilize the preprocessing tool, e.g. named entity recognizer, part-of-speech recognizer, lexical parser and coreference identifier etc., to incorporate more lexical and syntactical information into the feature vector.2. Encoding Layer encodes the representations by incorporating the context information or enriching the representation with desirable features for future use. For instance, a model can adopt bidirectional recurrent neural network to model the temporal interaction on both direction, recursive neural network BID54 (also known as TreeRNN) to model the compositionality and the recursive structure of language, or self-attention to model the long-term dependency on sentence. Different components of encoder can be combined to obtain a better sentence matrix representation.3. Interaction Layer creates an word-by-word interaction tensor by both premise and hypothesis representation matrix. The interaction can be modeled in different ways. A common approach is to compute the cosine similarity or dot product between each pair of feature vector. On the other hand, a high-order interaction tensor can be constructed with the outer product between two matrix representations.4. Feature Extraction Layer adopts feature extractor to extract the semantic feature from interaction tensor. The convolutional feature extractors, such as AlexNet BID36, VGG BID53, Inception BID55, ResNet BID28 and DenseNet BID31, proven work well on image recognition are completely compatible under such architecture. Unlike the work BID34 who employs 1-D sliding window, our CNN architecture allows 2-D kernel to extract semantic interaction feature from the word-by-word interaction between n-gram pair. Sequential or tree-like feature extractors are also applicable in the feature extraction layer. Output Layer decodes the acquired features to give prediction. Under the setting of NLI, the output layer predicts the confidence on each class. Here we introduce Densely Interactive Inference Network (DIIN) 1, which is a relatively simple instantiation of IIN but produces state-of-the-art performance on multiple datasets. Embedding Layer: For DIIN, we use the concatenation of word embedding, character feature and syntactical features. The word embedding is obtained by mapping token to high dimensional vector space by pre-trained word vector (840B GloVe). The word embedding is updated during training. As in BID35 BID37, we filter character embedding with 1D convolution kernel. The character convolutional feature maps are then max pooled over time dimension for each token to obtain a vector. The character features supplies extra information for some out-ofvocabulary (OOV) words. Syntactical features include one-hot part-of-speech (POS) tagging feature and binary exact match (EM) feature. The EM value is activated if there are tokens with same stem or lemma in the other sentence as the corresponding token. The EM feature is simple while found useful as in reading comprehension task BID21. In analysis section, we study how EM feature helps text understanding. Now we have premise representation P ∈ R p×d and hypothesis representation H ∈ R h×d, where p refers to the sequence length of premise, h refers to the sequence length of hypothesis and d means the dimension of both representation. The 1-D convolutional neural network and character features weights share the same set of parameters between premise and hypothesis. Encoding Layer: In the encoding layer, the premise representation P and the hypothesis representation H are passed through a two-layer highway network, thus havingP ∈ R p×d and H ∈ R h×d for new premise representation and new hypothesis representation. These new representation are then passed to self-attention layer to take into account the word order and context information. Take premise as example, we model self-attention by DISPLAYFORM0 whereP i is a weighted summation ofP. We choose α(a, DISPLAYFORM1, where w a ∈ R 3d is a trainable weight, • is element-wise multiplication, [;] is vector concatenation across row, and the implicit multiplication is matrix multiplication. Then bothP andP are fed into a semantic composite fuse gate (fuse gate in short), which acts as a skip connection. The fuse gate is implemented as DISPLAYFORM2 DISPLAYFORM3 DISPLAYFORM4 DISPLAYFORM5 where DISPLAYFORM6 We do the same operation on hypothesis representation, thus havingH. The weights of intraattention and fuse gate for premise and hypothesis are not shared, but the difference between the weights of are penalized. The penalization aims to ensure the parallel structure learns the similar functionality but is aware of the subtle semantic difference between premise and hypothesis. Interaction Layer: The interaction layer models the interaction between premise encoded representation P enc and hypothesis encoded representation H enc as follows: DISPLAYFORM7 whereP i is the i-th row vector ofP, andH j is the j-th row vector ofH. Though there are many implementations of interaction, we find β(a, b) = a • b very useful. Feature Extraction Layer: We adopt DenseNet BID31 as convolutional feature extractor in DIIN. Though our experiments show ResNet BID28 works well in the architecture, we choose DenseNet because it is effective in saving parameters. One interesting observation with ResNet is that if we remove the skip connection in residual structure, the model does not converge at all. We found batch normalization delays convergence without contributing to accuracy, therefore we does not use it in our case. A ReLU activation function is applied after all convolution unless otherwise noted. Once we have the interaction tensor I, we use a convolution with 1 × 1 kernel to scale down the tensor in a ratio, η, without following ReLU. If the input channel is k then the output channel is f loor(k × η). Then the generated feature map is feed into three sets of Dense block BID31 and transition block pair. The DenseNet block contains n layers of 3 × 3 convolution layer with growth rate of g. The transition layer has a convolution layer with 1 × 1 kernel for scaling down purpose, followed by a max pooling layer with stride 2. The transition scale down ratio in transition layer is θ. Output Layer: DIIN uses a linear layer to classify final flattened feature representation to three classes. In this section, we present the evaluation of our model. We first perform quantitative evaluation, comparing our model with other competitive models. We then conduct some qualitative analyses to understand how DIIN achieve the high level understanding through interaction. Here we introduce three datasets we evaluate our model on. The evaluation metric for all dataset is accuracy. SNLI Stanford Natural Language Inference (SNLI; Bowman et al. 2015) has 570k human annotated sentence pairs. The premise data is draw from the captions of the Flickr30k corpus, and the hypothesis data is manually composed. The labels provided in are "entailment", "neutral', "contradiction" and "-". "-" shows that annotators cannot reach consensus with each other, thus removed during training and testing as in other works. We use the same data split as in.MultiNLI Multi-Genre NLI Corpus (MultiNLI;) has 433k sentence pairs, whose collection process and task detail are modeled closely to SNLI. The premise data is collected from maximally broad range of genre of American English such as written non-fiction genres (SLATE, OUP, GOVERNMENT, VERBATIM, TRAVEL), spoken genres (TELEPHONE, FACE-TO-FACE), less formal written genres (FICTION, LETTERS) and a specialized one for 9/11. Half of these selected genres appear in training set while the rest are not, creating in-domain (matched) and cross-domain (mismatched) development/test sets. We use the same data split as provided by. Since test set labels are not provided, the test performance is obtained through submission on Kaggle.com 2. Each team is limited to two submissions per day. Quora question pair Quora question pair dataset contains over 400k real world question pair selected from Quora.com. A binary annotation which stands for match (duplicate) or not match (not duplicate) is provided for each question pair. In our case, duplicate question pair can be interpreted as entailment relation and not duplicate as neutral. We use the same split ratio as mentioned in. We implement our algorithm with Tensorflow BID14 framework. An Adadelta optimizer with ρ as 0.95 and as 1e−8 is used to optimize all the trainable weights. The initial learning rate is set to 0.5 and batch size to 70. When the model does not improve best indomain performance for 30,000 steps, an SGD optimizer with learning rate of 3e−4 is used to help model to find a better local optimum. Dropout layers are applied before all linear layers and after word-embedding layer. We use an exponential decayed keep rate during training, where the initial keep rate is 1.0 and the decay rate is 0.977 for every 10,000 step. We initialize our word embeddings with pre-trained 300D GloVe 840B vectors BID48 while the out-of-vocabulary word are randomly initialized with uniform distribution. The character embeddings are randomly initialized with 100D. We crop or pad each token to have 16 characters. The 1D convolution kernel size for character embedding is 5. All weights are constraint by L2 regularization, and the L2 regularization at step t is calculated as follows: DISPLAYFORM0 where L2F ullRatio determines the maximum L2 regularization ratio, and L2F ullStep determines at which step the maximum L2 regularization ratio would be applied on the L2 regularization. We choose L2F ullRatio as 0.9e − 5 and L2F ullStep as 100,000. 72.3 72.1 4. Gated-Att BiLSTM BID23 73.2 73.6 5. Shorcut-Stacked encoder BID46 74 Table 2: MultiNLI .difference of two encoder weights is set to 1e − 3. For a dense block in feature extraction layer, the number of layer n is set to 8 and growth rate g is set to 20. The first scale down ratio η in feature extraction layer is set to 0.3 and transitional scale down ratio θ is set to 0.5. The sequence length is set as a hard cutoff on all experiments: 48 for MultiNLI, 32 for SNLI and 24 for Quora Question Pair Dataset. During the experiments on MultiNLI, we use 15% of data from SNLI as in. We select the parameter by the best run of development accuracy. Our ensembling approach considers the majority vote of the predictions given by multiple runs of the same model under different random parameter initialization. We compare our with all other published systems in Table 2. Besides ESIM, the state-of-theart model on SNLI, all other models appear at RepEval 2017 workshop. RepEval 2017 workshop requires all submitted model to be sentence encoding-based model therefore alignment between sentences and memory module are not eligible for competition. All models except ours share one common feature that they use LSTM as a essential building block as encoder. Our approach, without using any recurrent structure, achieves the new state-of-the-art performance of 80.0%, exceeding current state-of-the-art performance by more than 5%. Unlike the observation from, we find the out-of-domain test performance is consistently lower than in-domain test performance. Selecting parameters from the best in-domain development accuracy partially contributes to this . In TAB4, we compare our model to other model performance on SNLI. Experiments are sentence encoding based model. Bowman et al. FORMULA0 provides a BiLSTM baseline. adopts two layer GRU encoder with pre-trained "skip-thoughts" vectors. To capture sentence-level semantics, use tree-based propose a stack-augmented parser-interpreter neural network (SPINN) which incorporates parsing information in a sequential manner. uses intra-attention on top of BiLSTM to generate sentence representation, and proposes an memory augmented neural network to encode the sentence. The next group of model, experiments BID5 BID6 BID7 BID8 BID9 BID10 BID11 BID12 BID13, uses cross sentence feature. aligns each sentence word-by-word with attention on top of LSTMs. enforces cross sentence attention word-by-word matching with the proprosed mL-STM model. proposes long short-term memory-network(LSTMN) with deep attention fusion that links the current word to previous word stored in memory. decomposes the task into sub-problems and conquer them respectively. proposes neural tree indexer, a full n-ary tree whose subtrees can be overlapped. Re-read LSTM proposed by considers the attention vector of one sentence as the inner-state of LSTM for another sentence. propose a sequential model that infers locally, and a ensemble with tree-like inference module that further improves performance. We show our model, DIIN, achieves state-of-the-art performance on the competitive leaderboard. Table 4: Quora question dataset . First six rows are copied from and next two rows from BID56. In this subsection, we evaluate the effectiveness of our model for paraphrase identification as natural language inference task. Other than our baselines, we compare with and BID56. BIMPM models different perspective of matching between sentence pair on both direction, then aggregates matching vector with LSTM. DECATT word and DECATT char uses automatically collected in-domain paraphrase data to noisy pretrain n-gram word embedding and ngram subword embedding correspondingly on decomposable attention model proposed by. In Table 4, our experiment shows DIIN has better performance than all other models and an ensemble score is higher than the former best for more than 1 percent. Ablation Study We conduct a ablation study on our base model to examine the effectiveness of each component. We study our model on MultiNLI dataset and we use Matched validation score as the standard for model selection. The is shown in Table 5. We studies how EM feature Table 5: Ablation study .contributes to the system. After removing the exact match binary feature, we find the performance degrade to 78.2 on matched score on development set and 78.0 on mismatched score. As observed in reading comprehension task BID21, the simple exact match feature does help the model to better understand the sentences. In the experiment 3, we remove the convolutional feature extractor and then model is structured as a sentence-encoding based model. The sentence representation matrix is max-pooled over time to obtain a feature vector. Once we have the feature vector p for premise and h for hypothesis, we use [p; h; |p − h|; p • h] as final feature vector to classify the relationship. We obtain 73.2 for matched score and 73.6 on mismatched data. The is competitive among other sentence-encoding based model. We further study how encoding layer contribute in enriching the feature space in interaction tensor. If we remove encoding layer completely, then we'll obtain a 73.5 for matched score and 73.2 for mismatched score. The demonstrate the feature extraction layer have powerful capability to capture the semantic feature. In experiment 5, we remove both self-attention and fuse gate, thus retaining only highway network. The improves to 77.7 and 77.3 respectively on matched and mismatched development set. However, in experiment 6, when we only remove fuse gate, to our surprise, the performance degrade to 73.5 for matched score and 73.8 for mismatched. On the other hand, if we use the addition of the representation after highway network and the representation after self-attention as skip connection as in experiment 7, the performance increase to 77.3 and 76.3. The comparison indicates self-attention layer makes the training harder to converge while a skip connection could ease the gradient flow for both highway layer and self-attention layer. By comparing the base model and the model the in experiment 6, we show that the fuse gate not only well serves as a skip connection, but also makes good decision upon which information the fuse for both representation. To show that dense interaction tensor contains more semantic information, we replace the dense interaction tensor with dot product similarity matrix between the encoded representation of premise and hypothesis. The shows that the dot product similarity matrix has an inferior capacity of semantic information. Another dimensionality study is provided in supplementary material. In experiment 9, we share the encoding layer weight, and the decrease from the baseline. The shows that the two set of encoding weights learn the subtle difference between premise and hypothesis. Error analysis To analyze the model prediction, we use annotated subset of development set provided by that consists of 1,000 examples each tagged with zero or more following tags:• CONDITIONAL: whether the sentence contains a conditional.• WORD OVERLAP: whether both sentences share more than 70% of their tokens.• NEGATION: whether a negation shows up in either sentence.• ANTO: whether two sentences contain antonym pair.• LONG SENTENCE: whether premise or hypothesis is longer than 30 or 16 tokens respectively.• TENSE DIFFERENCE: whether any verb in two sentences uses different tense.• ACTIVE/PASSIVE: whether there is an active-to-passive (or vice versa) transformation from the premise to the hypothesis. Table 6: MultiNLI .• PARAPHRASE: whether the two sentences are close paraphrases • QUANTITY/TIME REASONING: whether understanding the pair requires quantity or time reasoning.• COREF: Whether the hypothesis contains a pronoun or referring expression that needs to be resolved using the premise.• QUANTIFIER: Whether either sentence contains one of the following quantifier: much, enough, more, most, less, least, no, none, some, any, many, few, several, almost, nearly.• MODAL: Whether one of the following modal verbs appears in either sentence: can, could, may, might, must, will, would, should.• BELIEF: Whether one of the following belief verbs appear in either sentence: know, believe, understand, doubt, think, suppose, recognize, forget, remember, imagine, mean, agree, disagree, deny, promise. For more detailed descriptions, please resort to. The is shown in Table 6. We find DIIN is consistently better on sentence pair with WORD OVERLAP, ANTO, LONG SENTENCE, PARAPHRASE and BELIEF tags by a large margin. During investigation, we hypothesize exact match feature helps the model to better understand paraphrase, therefore we study the from second ablation ablation study where exact match feature is not used. Surprisingly, the model without exact model feature does not work worse on PARAPHRASE, instead, the accuracy on ANTO drops about 10%. DIIN is also work well on LONG SENTENCE, partially because the receptive field is large enough to cover all tokens. Visualization We also visualize the hidden representation from interaction tensor I and the feature map from first dense block in Figure 2. We pick a sentence pair whose premise is "South Carolina has no referendum right, so the Supreme Court canceled the vote and upheld the ban." and hypothesis is "South Carolina has a referendum right, so the Supreme Court was powerless over the state.". The upper row of figures are sampled from hidden representation of interaction tensor I. We observe the values of neurons are highly correlated row-wise and column-wise in the interaction tensor I and different channel of hidden representation shows different aspect of interaction. Though in certain channel same words, "referendum", or phrases, "supreme court", cause activation, different word or phrase pair, such as "ban" and "powerless over", also cause activation in other activation. It shows the model's strong capacity of understanding text in different perspective. The lower row of Figure 2 shows the feature map from first dense block. After being convolved from the interaction tensor and previous feature map, new feature maps shows activation in different position, demonstrating different semantic features are found. The first figure in the lower row has similar pattern as normal attention weight whereas others has no obvious pattern. Different channels of feature maps indicate different kinds of semantic feature. Figure 2: A visualization of hidden representation. The premise is "South Carolina has no referendum right, so the Supreme Court canceled the vote and upheld the ban." and the hypothesis is "South Carolina has a referendum right, so the Supreme Court was powerless over the state.". The upper row are sampled from interaction tensor I and the lower row are sample from the feature map of first dense block. We use viridis colormap, where yellow represents activation and purple shows the neuron is not active. We show the interaction tensor (or attention weight) contains semantic information to understand the natural language. We introduce Interactive Inference Network, a novel class of architecture that allows the model to solve NLI or NLI alike tasks via extracting semantic feature from interaction tensor end-to-end. One instance of such architecture, Densely Interactive Inference Network (DIIN), achieves state-of-the-art performance on multiple datasets. By ablating each component in DIIN and changing the dimensionality, we show the effectiveness of each component in DIIN.Though we have the initial exploration of natural language inference in interaction space, the full potential is not yet clear. We will keep exploring the potential of interaction space. Incorporating common-sense knowledge from external resources such as knowledge base to leverage the capacity of the mode is another research goal of ours.

Represent the TLDR for finding the corresponding scientific article

show multi-channel attention weight contains semantic feature to solve natural language inference task.

represent this text to find a much much longer article about it with say hundreds-thousands of words

A fully connected architecture is used to produce word embeddings from character representations, outperforms traditional embeddings and provides insight into sparsity and dropout.