Question: Represent the text to find a very short summary (1-2 sentences max)!

Question: CNNs are widely successful in recognizing human actions in videos, albeit with a great cost of computation. This cost is significantly higher in the case of long-range actions, where a video can span up to a few minutes, on average. The goal of this paper is to reduce the computational cost of these CNNs, without sacrificing their performance. We propose VideoEpitoma, a neural network architecture comprising two modules: a timestamp selector and a video classifier. Given a long-range video of thousands of timesteps, the selector learns to choose only a few but most representative timesteps for the video. This selector resides on top of a lightweight CNN such as MobileNet and uses a novel gating module to take a binary decision: consider or discard a video timestep. This decision is conditioned on both the timestep-level feature and the video-level consensus. A heavyweight CNN model such as I3D takes the selected frames as input and performs video classification. Using off-the-shelf video classifiers, VideoEpitoma reduces the computation by up to 50\% without compromising the accuracy. In addition, we show that if trained end-to-end, the selector learns to make better choices to the benefit of the classifier, despite the selector and the classifier residing on two different CNNs. Finally, we report state-of-the-art on two datasets for long-range action recognition: Charades and Breakfast Actions, with much-reduced computation. In particular, we match the accuracy of I3D by using less than half of the computation. A human can skim through a minute-long video in just a few seconds, and still grasp its underlying story . This extreme efficiency of the human visual and temporal information processing beggars belief. The unmatched trade-off between efficiency and accuracy can be attributed to visual attention -one of the hallmarks of the human cognitive abilities. This raises the question: can we build an efficient, yet effective, neural model to recognize minutes-long actions in videos? A possible solution is building efficient neural networks, which have a demonstrated record of success in the efficient recognition of images . Such models have been successful for recognizing short-range actions in datasets such as HMDB and UCF-101 , where analysis of only a few frames would suffice . In contrast, a long-range action can take up to a few minutes to unfold (a). Current methods fully process the long-range action video to successfully recognize it. Thus, for long-range actions, the major computational bottleneck is the sheer number of video frames to be processed. Another potential solution is attention. Not only it is biologically plausible, but also it is used in a wide spectrum of computer vision tasks, such as image classification, semantic segmentation , action recognition and temporal localization . Attention has also been applied to language understanding and graph modeling (Veličković et al., 2017). Most of these methods use soft-attention, where the insignificant visual signals are least attended to. However, such signals are still fully processed by the neural network and hence no reduction on the computation cost is obtained. Neural gating is a more conceivable choice to realize the efficiency, by completely dropping the insignificant visual signals. Recently, there has been a notable success in making neural gating differentiable . Neural gating is applied to conditional learning, and is used to gate network layers , convolutional channels , and more . That begs the question: can neural gating help in reducing the computational cost of recognizing minutes-long actions? That is to say, can we learn a gating mechanism to consider or discard video frames, conditioned on their video? Motivated by the aforementioned questions, we propose VideoEpitoma, a two-stage neural network for efficient classification of long-range actions without compromising the performance. The first stage is the timestep selector, in which, many timesteps of a long-range action are efficiently represented by lightweight CNN, such as MobileNet (; ;). Then, a novel gating module learns to select only the most significant timesteps -practically achieving the epitoma (Latin for summary) of this video. In the second stage, a heavyweight CNN, such as I3D , is used to effectively represent only the selected timesteps, followed by temporal modeling for the video-level recognition. This paper contributes the followings: i. VideoEpitoma, a neural network model for efficient recognition of long-range actions. The proposed model uses a novel gating module for timestep selection, conditioned on both the input frame and its context. ii. Off the shelf, our timestamp selector benefits video classification models and yields signification reduction in computation costs. We also show that if trained end-to-end, the timestep selector learns better gating mechanism to the benefit of the video classifier. iii. We present state-of-the-art on two long-range action recognition benchmarks: Charades and Breakfast Actions with significant reductions in the computational costs. Efficient Architectures. CNNs are the go-to solution when it comes to video classification. Thus, one prospective of reducing the computation of video recognition is to build efficient CNNs. Methods for pruning least important weights or filters were previously proposed. Careful design choices in very efficient 2D CNNs such as MobileNet and ShuffleNet . These 2D CNNs are extended to their 3D counterparts (ShuffleNet-3D and MobileNet-3D byKöpüklü et al. ) to learn spatiotemporal concepts for video classification. Neural architecture search is used to find the lightweight NasNet-Mobile . Long-range Actions Short-range actions in datasets such as Kinetics and UCF-101 have average length of 10 seconds. They can be practically classified with CNNs using as little as 10 frames per video, and in some cases, even 1 frame would suffice . Therefore, building efficient CNNs is a plausible choice to reduce computational cost of recognizing them. However, long-range videos (e.g. Charades and Breakfast Actions ) can take up to 5 minutes to unfold. Thus, requiring as many as a thousand frames (a; b) to be correctly classified. As such, analyzing all the frames using efficient CNNs can still be computationally expensive. In contrast, having a mechanism to select the most relevant frames can boost the efficiency. Therefore, this paper focuses on reducing the number of video frames needed for action recognition. Nevertheless, our work is orthogonal to prior works that focus on development of efficient CNN for action recognition. Conditional Computing. Another solution to reduce the computation is to dynamically route the compute graph of a neural network. The assumption is that not all input signals require the same amount of computation -some are complicated while others are seemingly easy. Thanks to categorical reparametarization , it becomes possible to discretize a continuous distribution, and effectively learn binary gating. In , a dynamical graph is build by gating the layers of a typical CNN. While in ), the gating is achieved on the level of convolutional channels. In the same vien, GaterNet proposes a separate gater network to learn binary gates for the backbone network. Differently, this paper focuses on gating the video frames themselves, to realize efficiency. Sampling of Video Frames. Several works discuss frame sampling for short-range videos. SCSampler learns a ranking score using trimmed v.s. untrimmed video segments. proposes a student-teacher model for trimmed video classification. In , an agent is trained with reinforcement to learn where to look next. However, frame sampling for long-range actions is fundamentally different from that of short-range. Unlike shortrange actions, in long-range actions, usually a much smaller proportion of timesteps are crucial for classification. As a , this paper focuses on frame selection for solely long-range actions, and it does not require any video-level annotation other than the video category itself. Then it temporally models these selected timesteps to arrive at the video-level feature, which is then classified. Figure 2: Bottom, the Timestep Selector learns concept kernels to represent the dominant visual concepts across the videos. Top, the gating module learns to select only a few timesteps according to their importance to the current video. Model Overview. VideoEpitoma consists of two stages: Timestep Selector and Video Classifier, see figure 1. The first stage is the Timestep Selector and consists of a lightweight CNN, LightNet, followed by a novel gating module, see figure 2. The purpose of this module is timestep gating, i.e. to take binary decision of considering or discarding each video timestep, based on how relevant it is to the video itself. The second stage is the video classifier. Its main purpose is to learn deep and discriminatory video-level representations for maximum classification accuracy. Thus, it resides on top of a heavyweight CNN, HeavyNet, followed by an off-the-shelf temporal layer for video-level representation, and a Multi-Layer Perceptron (MLP) for classification. Only the timesteps chosen by the first stage, i.e. the Timestep Selector, are considered by the second stage, i.e. the video classifier. The Timestep Selector. Conceptually speaking, a longrange action consists of few yet dominant and discriminative visual concepts, based on which, the video can be recognized (b; a). Take for example "Making Pancake". One can easily discriminate it by observing its dominant evidences "Pancake", "Eggs", "Pan", and "Stove". These evidences can be thought of latent concepts. To represent these concepts, we opt for learning a set of N concept kernels K = {k 1, k 2, ...k N}. K are randomly initialized and are part of the network parameters and learned during the training of the selector. Our concept kernels K are reminiscent of the nodes in VideoGraph (b) or the centroids in ActionVLAD . Once these concepts are learned, it becomes easier to efficiently summarize a long-range action. We transverse through a video of thousands timesteps and decide which of them to consider and which to discard, based on the similarity between the features of these timesteps and that of the latent concepts. Our assumption is that a lightweight representation of each timestep is sufficient for taking this decision. Thus, the selector depends on an efficient LightNet to represent these timesteps. Given a long-range video v of T timestep, each is represented as a feature x i ∈ R C×H×W using the LightNet, where C is the convolutional channels, H, W are the channel height and width, respectively. The Gating Module. The purpose of the gating module is to select the video timsteps, see figure 2 top. We start by comparing how relevant each timstep feature x i is to all of the concept kernels K ∈ R N×C using a dot product. The is the similarity scores, representing how relevant a timestep is to each of these concept kernels. Then we model the correlation between these similarity scores s i with a two-layer MLP with a single neuron in the output layer, denoted as α. Next, we need to convert the continuous variable α to a binary variable, such that it represents the decision of the gating module. For this, we make use of to discretize a continuous variable. Following the gating mechanism of , we add gumbel noise to α and follow with sigmoid activation and binary thresholding, arriving at the activated gating valueα. Then, each timestep feature x i is multiplied byα, to either select or discard it. A problem with the aforementioned gating mechanism is that during the feedforward, the classifier does not know which of the selected timesteps is more relevant than the other. As a remedy, we propose a different gating mechanism, see figure 2, top. First, a sigmoid non-linearity is applied to the gating value α to limit its lower-and upper-bound,α = sigmoid(α). Then, to achieve gating, we clipα below threshold 0.5. This modified activation function clipped sigmoid fits perfectly to the purpose of timestep gating due to 3 desirable properties, see figure 3. i. Being a relaxation for the step-function makes it differentiable. ii. Retaining the sigmoid value above the threshold means that the classifier gets the chance to know, out of the selected timesteps, which is relatively more important than the other. iii. Unlike ReLU, the sigmoid activation is upperbounded by 1, thus preventing a single timestep from dominating the others by being multiplied by unbounded gating valueα. Context Conditional Gating. Up till now, the selector learns to gate each timestep regardless of its context, i.e. the video itself. To achieve conditional gating, where both the timestep and its context affect the gating mechanism, we opt for a temporal modeling layer, self-attention , before the gating module, See figure 2, bottom. This temporal layer learns to correlate each timestep with all the others in the video before gating. For selecting timesteps during training, the gating module uses gated-sigmoid as the activation for the gating value α. It has some desirable properties. i. Unlike ReLU, having upper bound does not allow a timestep feature to dominate others. ii. Unlike sigmoid, being clipped allows the network to discard insignificant timesteps, i.e. those with gating values α < 0.5. In test time, we replace the gated-sigmoid with step-function for binary gating of timesteps. Sparse Selection. The last component of the selector is to enforce sparsity on timestep selection, i.e. choose as few timesteps as possible, yet retain the classification accuracy. Loosely speaking, the selector can simply cheat by predicting gating values α just higher than the threshold 0.5, ing in all gates opened and all timesteps selected. The selector has a natural tendency to such a behaviour, as the only loss used so far is that of classification. And the more timesteps used by the classifier, the better the classification accuracy. To prevent such a behaviour, we apply L 0 regularization to the gating valuesα to enforce sparsity on the selected timesteps. We note that the Sparsity regularization is necessary for a properly functioning gating mechanism. The Video Classifier The assumption of VideoEpitoma is that having efficiently selected the most crucial timesteps from the video using the LightNet and the selector, one can opt for a much more powerful HeavyNet to effectively classify the video. Thus, the second stage of VideoEpitoma is the video classifier, see figure 1. It takes as input only the subset T of timesteps chosen by the selector, T T. Each timestep is represented as a feature y i using HeavyNet. Following feature extraction, we use one layer of self-attention for temporal modeling to obtain a video-level representation, followed by a two-layer MLP for the final classification. Before training VideoEpitoma, all the CNN used, as LightNet and HeavyNet, are fine-tuned first on the videos of the dataset in hand. VideoEpitoma is trained with batch size 32 and for 100 epochs. We use Adam with learning rate 1e-3 and epsilon 1e-4. We use PyTorch and TensorFlow for our implementation. Our choice for the LightNet is MobileNetv3. As for the HeavyNet, we experiment with I3D , ShuffleNet3D and ResNet2D (the 50-layer version). Worth mentioning that in the gating module, and during the training phase, we use gumbel noise and clipped sigmoid to get the activated gating valueα, see figure 2. In the test phase, we don't use gumbel noise, and we use step-function, to get a binary gating value. Breakfast Actions Breakfast Actions is a dataset for long-range actions, depicting cooking activities. All in all, it contains 1712 videos, divided into 1357 and 335 for training and testing, respectively. The task is video recognition into 10 classes of making different breakfasts. Added to the video-level annotation, we are given temporal annotations of 48 one-actions. In our experiments, we only use the video-level annotation, and we do not use the temporal annotation of the one-actions. The videos are long-range, with the average length of 2.3 minutes per video. Which makes it ideal for testing the efficiency of recognizing long-range actions. The evaluation method is the accuracy. Charades Charades is a widely used benchmark for human action recognition. It is a diverse dataset with 157 action classes in total. The task is mult-label recognition, where each video is assigned to one or more action class. It is divided into 8k, 1.2k and 2k videos for training, validation and test splits, respectively, covering 67 hours. On average, each video spans 30 seconds, and is labeled with 6 and 9 actions for training and test splits, respectively. Thus, Charades meets the criteria of long-range actions. We use Mean Average Precision (mAP) for evaluation. One might raise an important question -will a Timestep Selector based on LightNet features benefit a classifier based on HeavyNet features, given the differences between the feature spaces of LightNet and HeavyNet? To answer this question, we construct an experiment of two steps on Breakfast. The first step is training a stand-alone selector. For this, we train VideoEpitoma to classify the videos of Breakfast, where we choose MobileNet for both LightNet and HeavyNet. During training, we randomly sample T = 32 timesteps from each video. Since MobileNet is a 2D CNN, a timestep here is practically a video frame. With the help of the L 0 regularization, the selector achieves sparse selection of timesteps, by as little as T = 16 without degrading the classification performance. The second step is testing how will the selector benefit off-the-shelf CNN classifiers. For this, we use different CNN classifiers, previously fine-tuned on Breakfast: I3D, ShuffleNet3D and ResNet2D. Then, we measure their performance using sampled T ∈ {1, 2, 4, 8, 16} timesteps from each video. We use different sampling methods: i. random, ii. uniform and iii. timestep selector. As discussed, the output of the timestep selector is a per-timestep binary valueα ∈ {0, 1} of whether to consider or discard this timestep. So, if T timesteps are processed by the selector, it is able to choose a subset T timesteps and discard the others, where T T. And to to evaluate the benefit of the selector, the off-the-self classifier then uses only T. As shown in figure 4, we observe that the stand-alone selector helps the off-the-shelf classifiers to retain their performance with a reduction of up to 50% of the timesteps. The same improvement is observed for three different CNN classifiers: I3D, ResNet2D and ShuffleNet3D. The main of this experiment is the following. To realize the efficient recognition of long-range actions, reducing the number of processed timesteps is far more rewarding than reducing the processing of each timestep. In other words, our Timestep Selector is able to reduce, by more than half, the computation of the already efficient ShuffleNet3D. See the appendix for full . Having demonstrated that a stand-alone selector can benefit off-shelf classifiers, we pose another question -is it possible to train VideoEpitoma end-to-end, given that the selector and the classifier operate on features from two different CNNs, LightNet and HeavyNet, with two different feature spaces. To answer this question, we do the following experiment. We train VideoEpitoma in an endto-end fashion, where we choose the efficient MobileNet as the LightNet of the selector. As for the HeavyNet of the classifer, we explore multiple choices: I3D, ShuffleNet3D and ResNet2D. Based on our experiments, a careful consideration is to align the timestep features of the 2D LightNet with that of the 3D HeavyNet. In a typical 3D HeavyNet, each timestep is a video snippet of m successive frames {f j, ...., f j+m}, represented as one timestep feature y i ∈ R C×H×W. Thus, the corresponding feature x i from the 2D LightNet has to be based on the middle frame of the snippet, i.e. frame f j+(m/2). c e r e a l s c o f f e e f r i e d e g g j u i c e m i l k p a n c a k e s a l a t s a Stand-alone End-to-end ResNet2D End-to-end I3D End-to-end ShuffleNet3D Figure 5: The ratio of selected timesteps for the action categories of Breakfast. When VideoEpitoma is trained end-to-end, the Timestep Selector learns a better selection to the benefit of the classifer. Notice that the selection ratio changes from stand-alone selector (red) to end-to-end training with the HeavyNets: ResNet2D (green) I3D (yellow) and ShuffleNet3D (blue). The findings of this experiment are as follows. Figure 5 shows the ratio of the selected timesteps by the selector for the videos of each action category of Breakfast. The ratios of the stand-alone (red) is changed when it is trained end-to-end with different HeavyNet: ResNet2D, (blue), I3D (yellow), and ShuffleNet3D (blue). Also, we observe that the choices for the selector when the HeavyNet is 3D CNN tends to agree, relgardless of which 3D CNN is used. Between yellow and blue, we see agreement for 8 of 10 actions. However, the choices tend to vary between 2D and 3D as HeavyNet. Between green and yellow, there is agreement for 3 our of 10 actions. From this experiment, we conclude that, the gating module, depending on LightNet features, learns to select better timesteps to the benefit of the HeavyNet classifier. Gating irrelevant visual evidences is of a great importance in recognizing long-range actions. For example, when discriminating two action categories "Making Pancake" and "Preparing Coffee", we want to make a gating decision for the visual evidences of "Knife" and "Pan". It is better to discard "Knife" as it is irrelevant to both of actions -this is called frame gating. However, the visual evidence of "Pan" is relevant to only "Making Pancake". Thus, it's optimal to consider it only if the action is "Making Pancake" and discarding it otherwise -this is called context gating. In the Timestep Selector, see figure 2 bottom, we use a temporal modeling layer before the gating module. It enables the correlation between a timestep feature, and the video context, i.e. the other timestep features. As a , the gating mechanism becomes conditioned on both the timestep and the video context. To verify this assumption, we conduct an ablation study. We train a variant of the Timestep Selector without the temporal modeling layer, which makes the gating mechanism conditioned on only the timestep feature. In the Timestep Selector, the gating meachnism is conditioned on both the timestep-level feature and the video-level context, which is a better conditional gating. If the gating is only frame-conditioned, the ratios of the selected timesteps for action categories have small variance. Which means the gating is less dependent on the context, i.e. the action category. On contrary, the notice a big variance for the frame and contextconditioned. The gating becomes more dependent on the action category when selecting the timesteps. We observe a drop in performance when using this variant of the Timestep Selector. The reason is that when the gating is conditioned only on the timestep feature, it acts as a saliency selector. That is to say, the gating discards only the frames not related to any of the action categories of the dataset. Figure 6, left, shows the ratio of selected timesteps for each action categories of Breakfast. The frame-conditioned gating (blue) tends to select similar ratios regardless of the category. In contrast, we see more diverse ratios for the timestep and context-conditioned gating. Figure 6, right, shows the ratio variances for the two gating mechanisms. The much higher variance for context gating means that it is more dependent on the action category than the frame gating. We conclude that the cost of selecting timestep using LightNet is marginal to that of the HeavyNet and classifier. When it comes to the recognition of long-range actions, the golden rule is the more timesteps the better the accuracy, and the heavier the computation. But given the huge redundancies of the visual evidences in these timesteps, there is a tradeoff between accuracy and computation. In this experiment, we explore what is effect of this tradeoff on VideoEpitoma, and we compare against off-the-shelf CNNs. Figure 7 shows this tradeoff for three different CNNs: I3D, ResNet2D and ShuffleNet3D. While table 1 details the exact computational budget of VideoEpitoma v.s. the competing CNN. The of this experiment is twofold. First, when it comes to classifying the minuteslong actions, classifying a handful of carefully selected timesteps, using VideoEpitoma, is far more rewarding solution than efficiently all of them, using for example ShuffleNet3D. Second, the cost of selecting these timesteps can be significantly reduced by using a lightweight 2D CNN, as MobileNet. (RNet2D), ii. ShuffleNet3D (SNet3D) and iii. I3D. The computational cost of LightNet and the gating module is marginal compared to that of the HeavyNet. In addition, our selector retains the performance of the HeavyNet but with using half of the timesteps and almost half of the computational cost. Our final experiment is to experiment how VideoEpitoma would fair against off-the-shelf CNN for recognizing the multi-label action videos of Charades. Charades differs from Breakfast in two ways. i Videos of Charades are mid-range, with 0.5 minutes as average length, compared to 2 minutes of Breakfast. ii Charades is multi-label classifications, with 7 labels per video, and 157 labels in total. Breakfast is single-label classification, with 10 labels in total. Due to these two differences, it is harder to select unrelated timesteps from the videos of Charades than Breakfast -most of the timesteps are already relevant to recognizing the mid-range videos of Charades. Still, VideoEpitoma outperforms the off-the-shelf ResNet2D, at different time scales, see figure 8. In this paper, we proposed VideoEpitoma, a neural model for efficient recognition of long-range actions in videos. We stated the fundamental differences between long-range actions and their shortrange counterparts (a; b). And we highlighted how these differences influenced our way of find a solution for an efficient recognition of such videos. The outcome of this paper is VideoEpitoma, a neural model with the ability to retain the performance of off-the-shelf CNN classifiers at a fraction of the computational budget. This paper concludes the following. Rather than building an efficient CNN video classifier, we opted for an efficient selection of the most salient parts of the video, followed by an effective classification of only these salient parts. For a successful selection, we proposed a novel gating module, able to select timesteps conditioned on their importance to their video. We experimented how this selection benefits off-the-shelf CNN classifiers. Futher more, we showed how VideoEpitoma, i.e. both the selector and the classifier, improves even further when trained end-to-end. Finally, we experimented VideoEpitoma on two benchmarks for longrange actions. We compared against realted methods to hightight the efficiency of videoEpitoma for saving the computation, and its effectiveness of recognizing the long-range actions.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

Efficient video classification using frame-based conditional gating module for selecting most-dominant frames, followed by temporal modeling and classifier.

represent the tldr for finding the corresponding scientific article

In this paper we introduced an algorithm to learn the connectivity of deep multi-branch networks. The approach is evaluated on image categorization where it consistently yields accuracy gains over state-of-the-art models that use fixed connectivity.


Question: Represent the scientific article to find a short TLDR

Question: Human annotation for syntactic parsing is expensive, and large resources are available only for a fraction of languages. A question we ask is whether one can leverage abundant unlabeled texts to improve syntactic parsers, beyond just using the texts to obtain more generalisable lexical features (i.e. beyond word embeddings). To this end, we propose a novel latent-variable generative model for semi-supervised syntactic dependency parsing. As exact inference is intractable, we introduce a differentiable relaxation to obtain approximate samples and compute gradients with respect to the parser parameters. Our method (Differentiable Perturb-and-Parse) relies on differentiable dynamic programming over stochastically perturbed edge scores. We demonstrate effectiveness of our approach with experiments on English, French and Swedish. A dependency tree is a lightweight syntactic structure exposing (possibly labeled) bi-lexical relations between words BID77 BID24, see Figure 1. This representation has been widely studied by the NLP community leading to very efficient state-of-the-art parsers BID30 BID12 BID43, motivated by the fact that dependency trees are useful in downstream tasks such as semantic parsing BID66, machine translation BID11 BID4, information extraction BID9 BID42, question answering BID8 and even as a filtering method for constituency parsing BID34, among others. Unfortunately, syntactic annotation is a tedious and expensive task, requiring highly-skilled human annotators. Consequently, even though syntactic annotation is now available for many languages, the datasets are often small. For example, 31 languages in the Universal Dependency Treebank, 1 the largest dependency annotation resource, have fewer than 5,000 sentences, including such major languages as Vietnamese and Telugu. This makes the idea of using unlabeled texts as an additional source of supervision especially attractive. In previous work, before the rise of deep learning, the semi-supervised parsing setting has been mainly tackled with two-step algorithms. On the one hand, feature extraction methods first learn an intermediate representation using an unlabeled dataset which is then used as input to train a supervised parser BID35 BID83 BID7 BID73. On the other hand, the self-training and co-training methods start by learning a supervised parser that is then used to label extra data. Then, the parser is retrained with this additional annotation BID68 BID25 BID50. Nowadays, unsupervised feature extraction is achieved in neural parsers by the means of word embeddings BID55 BID65. The natural question to ask is whether one can exploit unlabeled data in neural parsers beyond only inducing generalizable word representations. Figure 1: Dependency tree example: each arc represents a labeled relation between the head word (the source of the arc) and the modifier word (the destination of the arc). The first token is a fake root word. Our method can be regarded as semi-supervised Variational Auto-Encoder . Specifically, we introduce a probabilistic model (Section 3) parametrized with a neural network (Section 4). The model assumes that a sentence is generated conditioned on a latent dependency tree. Dependency parsing corresponds to approximating the posterior distribution over the latent trees within this model, achieved by the encoder component of VAE, see Figure 2a. The parameters of the generative model and the parser (i.e. the encoder) are estimated by maximizing the likelihood of unlabeled sentences. In order to ensure that the latent representation is consistent with treebank annotation, we combine the above objective with maximizing the likelihood of gold parse trees in the labeled data. Training a VAE via backpropagation requires marginalization over the latent variables, which is intractable for dependency trees. In this case, previous work proposed approximate training methods, mainly differentiable Monte-Carlo estimation BID27 BID67 and score function estimation, e.g. REINFORCE BID80. However, REINFORCE is known to suffer from high variance BID56. Therefore, we propose an approximate differentiable Monte-Carlo approach that we call Differentiable Perturb-and-Parse (Section 5). The key idea is that we can obtain a differentiable relaxation of an approximate sample by perturbing weights of candidate dependencies and performing structured argmax inference with differentiable dynamic programming, relying on the perturbed scores. In this way we bring together ideas of perturb-and-map inference BID62 BID45 and continuous relaxation for dynamic programming BID53. Our model differs from previous works on latent structured models which compute marginal probabilities of individual edges BID26; BID41. Instead, we sample a single tree from the distribution that is represented with a soft selection of arcs. Therefore, we preserve higher-order statistics, which can then inform the decoder. Computing marginals would correspond to making strong independence assumptions. We evaluate our semi-supervised parser on English, French and Swedish and show improvement over a comparable supervised baseline (Section 6).Our main contributions can be summarized as follows: we introduce a variational autoencoder for semi-supervised dependency parsing; we propose the Differentiable Perturb-and-Parse method for its estimation; we demonstrate the effectiveness of the approach on three different languages. In short, we introduce a novel generative model for learning latent syntactic structures. A dependency is a bi-lexical relation between a head word (the source) and a modifier word (the target), see Figure 1. The set of dependencies of a sentence defines a tree-shaped structure. 2 In the parsing problem, we aim to compute the dependency tree of a given sentence. Formally, we define a sentence as a sequence of tokens (words) from vocabulary W. We assume a one-to-one mapping between W and integers 1... |W|. Therefore, we write a sentence of length n as a vector of integers s of size n + 1 with 1 ≤ s i ≤ |W| and where s 0 is a special root symbol. A dependency tree of sentence s is a matrix of booleans T ∈ {0, 1} (n+1)×(n+1) with T h,m = 1 meaning that word s h is the head of word s m in the dependency tree. DISPLAYFORM0 Figure 2: (a) Illustration of our probabilistic model with random variables s, T and z for sentences, dependency trees and sentence embeddings, respectively. The gray area delimits the latent space. Solid arcs denote the generative process, dashed arcs denotes posterior distributions over the latent variables. (b) Stochastic computation graph. (c) Illustration of the decoder when computing the probability distribution of s 4, the word at position 4. Dashed arcs at the bottom represent syntactic dependencies between word at position 4 and previous positions. At each step, the LSTM takes as input an embedding of the previous word (s 0 is a special start-of-sentence symbol). Then, the GCN combines different outputs of the LSTM by transforming them with respect to their syntactic relation with the current position. Finally, the probability of s 4 is computed via the softmax function. More specifically, a dependency tree T is the adjacency matrix of a directed graph with n + 1 vertices v 0... v n. A matrix T is a valid dependency tree if and only if this graph is a v 0 -rooted spanning arborescence, 3 i.e. the graph is connected, each vertex has at most one incoming arc and the only vertex without incoming arc is v 0. A dependency tree is projective if and only if, for each arc v h → v m, if h < m (resp. m < h) then there exists a path with arcs T from v h to each vertex v k such that h < k < m (resp. m < k < h). From a linguistic point of view, projective dependency trees combine contiguous phrases (sequence of words) only. Intuitively, this means that we can draw the dependency tree above the sentence without crossing arcs. Given a sentence s, an arc-factored dependency parser computes the dependency tree T which maximizes a weighting function f (T ; W) = h,m T h,m W h,m, where W is a matrix of dependency (arc) weights. This problem can be solved with a O(n 2) time complexity BID75 BID51. If we restrict T to be a projective dependency tree, then the optimal solution can be computed with a O(n 3) time complexity using dynamic programming BID16. Restricting the search space to projective trees is appealing for treebanks exhibiting this property (either exactly or approximately): they enforce a structural constraint that can be beneficial for accuracy, especially in a low-resource scenario. Moreover, using a more restricted search space of potential trees may be especially beneficial in a semi-supervised scenario: with a more restricted space a model is less likely to diverge from a treebank grammar and capture non-syntactic phenomena. Finally, Eisner's algorithm BID16 can be described as a deduction system BID64, a framework that unifies many parsing algorithms. As such, our methodology could be applied to other grammar formalisms. For all these reasons, in this paper, we focus on projective dependency trees only. We now turn to the learning problem, i.e. estimation of the matrix W. We assume that we have access to a set of i.i.d. labeled sentences L = {s, T, . . .} and a set of i.i.d. unlabeled sentences U = {s, . . .}. In order to incorporate unlabeled data in the learning process, we introduce a generative model where the dependency tree is latent (Subsection 3.1). As such, we can maximize the likelihood of observed sentences even if the ground-truth dependency tree is unknown. We learn the parameters of this model using a variational Bayes approximation (Subsection 3.2) augmented with a discriminative objective on labeled data (Subsection 3.3). Under our probabilistic model, a sentence s is generated from a continuous sentence embedding z and with respect to a syntactic structure T. We formally define the generative process of a sentence of length n as: DISPLAYFORM0 This Bayesian network is shown in Figure 2a. In order to simplify notation, we omit conditioning on n in the following. T and z are latent variables and p(s|T, z) is the conditional likelihood of observations. We assume that the priors p(T) and p(z) are the uniform distribution over projective trees and the multivariate standard normal distribution, respectively. The true distribution underlying the observed data is unknown, so we have to learn a model p θ (s|T, z) parametrized by θ that best fits the given samples: DISPLAYFORM1 Then, the posterior distribution of latent variables p θ (T, z|s) models the probability of underlying representations (including dependency trees) with respect to a sentence. This conditional distribution can be written as: DISPLAYFORM2 In the next subsection, we explain how these two quantities can be estimated from data. Computations in Equation 1 and Equation 2 require marginalization over the latent variables: DISPLAYFORM0 which is intractable in general. We rely on the Variational Auto-Encoder (VAE) framework to tackle this challenge BID27 BID67. We introduce a variational distribution q φ (T, z|s) which is intended to be similar to p θ (T, z|s). More formally, we want KL [q φ (T, z|s) p θ (T, z|s)] to be as small as possible, where KL is the Kulback-Leibler (KL) divergence. Then, the following equality holds: DISPLAYFORM1 where log p θ (s) is called the evidence. The KL divergence is always positive, therefore by removing the last term we have: DISPLAYFORM2 where the right-hand side is called the Evidence Lower Bound (ELBO). By maximizing the ELBO term, the divergence KL [q φ (T, z|s) p θ (T, z|s)] is implicitly minimized. Therefore, we define a surrogate objective, replacing the objective in Equation 1: DISPLAYFORM3 The ELBO in Equation 4 has two components. First, the KL divergence with the prior, which usually has a closed form solution. For the distribution over dependency trees, it can be computed with the semiring algorithm of BID38. Second, the non-trivial term DISPLAYFORM4 During training, Monte-Carlo method provides a tractable and unbiased estimation of the expectation. Note that a single sample from q φ (T, z|s) can be understood as encoding the observation into the latent space, whereas regenerating a sentence from the latent space can be understood as decoding. However, training a VAE requires the sampling process to be differentiable. In the case of the sentence embedding, we follow the usual setting and define q φ (z|s) as a diagonal Gaussian: backpropagation through the the sampling process z ∼ q φ (z|s) can be achieved thanks to the reparametrization trick BID27 BID67. Unfortunately, this approach cannot be applied to dependency tree sampling T ∼ q φ (T |s). We tackle this issue in Section 5. VAEs are a convenient approach for semi-supervised learning and have been successfully applied in NLP BID33 BID81 BID85 BID82. In this scenario, we are given the dependency structure of a subset of the observations, i.e. T is an observed variable. Then, the supervised ELBO term is defined as: DISPLAYFORM0 Note that our end goal is to estimate the posterior ditribution over dependency trees q φ (T |s), i.e. the dependency parser, which does not appear in the supervised ELBO. We want to explicitly use the labeled data in order to learn the parameters of this parser. This can be achieved by adding a discriminative training term to the overall loss. The loss function for training a semi-supervised VAE is: DISPLAYFORM0 where the first term is the standard loss for supervised learning of log-linear models BID23 BID36 ). In this section, we describe the neural parametrization of the encoder distribution q φ (Subsection 4.1) and the decoder distribution p θ (Subsection 4.2). A visual representation is given in Figure 2b. We factorize the encoder as q φ (T, z|s) = q φ (T |s)q φ (z|s). The categorical distribution over dependency trees is parametrized by a log-linear model BID36 where the weight of an arc is given by the neural network of BID30.The sentence embedding model is specified as a diagonal Gaussian parametrized by a LSTM, similarly to the seq2seq framework BID72 BID5. That is: DISPLAYFORM0 where m and v are mean and variance vectors, respectively. We use an autoregressive decoder that combines an LSTM and a Graph Convolutional Network (; . The LSTM keeps the history of generated words, while the GCN incorporate information about syntactic dependencies. The hidden state of the LSTM is initialized with latent variable z (the sentence embedding). Then, at each step 1 ≤ i ≤ n, an embedding associated with word at position i − 1 is fed as input. A special start-of-sentence symbol embedding is used at the first position. Let o i be the hidden state of the LSTM at position i. The standard seq2seq architecture uses this vector to predict the word at position i. Instead, we transform it in order to take into account the syntactic structure described by the latent variable T. Due to the autoregressive nature of the decoder, we can only take into account dependencies T h,m such that h < i and m < i. Before being fed to the GCN, the output of the LSTM is fed to distinct multi-layer perceptrons 6 that characterize syntactic relations: if s h is the head of s i, o h is transformed with MLP, if s m is a modifier of s i, o m is transformed with MLP, and lastly o i is transformed with MLP. Formally, the GCN is defined as follows: DISPLAYFORM0 The output vector g i is then used to estimate the probability of word s i. The neural architecture of the decoder is illustrated on Figure 2c. Encoder-decoder architectures are usually straightforward to optimize with the back-propagation algorithm BID40 BID37 ) using any autodiff library. Unfortunately, our VAE contains stochastic nodes that can not be differentiated efficiently as marginalization is too expensive or intractable (see Figure 2b for the list of stochastic nodes in our computation graph). BID27 and BID67 proposed to rely on a Monte-Carlo estimation of the gradient. This approximation is differentiable because the sampling process is moved out of the backpropagation path. In this section, we introduce our Differentiable Perturb-and-Parse operator to cope with the distribution over dependency trees. Firstly, in Subsection 5.1, we propose an approximate sampling process by computing the best parse tree with respect to independently perturbed arc weights. Secondly, we propose a differentiable surrogate of the parsing algorithm in Subsection 5.2. Sampling from a categorical distributions can be achieved through the Gumbel-Max trick BID20 BID44. 8 Unfortunately, this reparametrization is difficult to apply when the discrete variable can take an exponential number of values as in Markov Random Fields (MRF). BID62 proposed an approximate sampling process: each component is perturbed independently. Then, standard MAP inference algorithm computes the sample. This technique is called perturb-and-map. Arc-factored dependency parsing can be expressed as a MRF where variable nodes represent arcs, singleton factors weight arcs and a fully connected factor forces the variable assignation to describe a valid dependency tree BID71. Therefore, we can apply the perturb-and-map method to dependency tree sampling: DISPLAYFORM0 where G is the Gumbel distribution, that is sampling matrix P is equivalent to setting P i,j = − log(− log U i,j)) where U i,j ∼ Uniform.Algorithm 1 This function search the best split point for constructing an element given its span. b is a one-hot vector such that b i−k = 1 iff k is the best split position. DISPLAYFORM1 s ← null-initialized vec. of size j − i 3:for i ≤ k < j do 4: DISPLAYFORM2 b ← ONE-HOT-ARGMAX(s) 6: DISPLAYFORM3 has contributed the optimal objective, this function sets T i,j to 1. Then, it propagates the contribution information to its antecedents. 1: function BACKTRACK-URIGHT(i, j, T) 2: DISPLAYFORM4 for i ≤ k < j do 5: DISPLAYFORM5 6: DISPLAYFORM6 The (approximate) Monte-Carlo estimation of the expectation in Equation 3 is then defined as: DISPLAYFORM7 where denotes a Monte-Carlo estimation of the gradient, P ∼ G is sampled in the last line and EISNER is an algorithm that compute the projective dependency tree with maximum (perturbed) weight BID16. Therefore, the sampling process is outside of the backpropagation path. Unfortunately, the EISNER algorithm is built using ONE-HOT-ARGMAX operations that have illdefined partial derivatives. We propose a differentiable surrogate in the next section. We now propose a continuous relaxation of the projective dependency parsing algorithm. We start with a brief outline of the algorithm using the parsing-as-deduction formalism, restricting this presentation to the minimum needed to describe our continuous relaxation. We refer the reader to BID16 for an in-depth presentation. The parsing-as-deduction formalism provides an unified presentation of many parsing algorithms BID64 BID70. In this framework, a parsing algorithm is defined as a deductive system, i.e. as a set of axioms, a goal item and a set of deduction rules. Each deduced item represents a sub-analysis of the input. Regarding implementation, the common way is to rely on dynamic programming: items are deduced in a bottom-up fashion, from smaller sub-analyses to large ones. To this end, intermediate are stored in a global chart. For projective dependency parsing, the algorithm builds a chart whose items are of the form DISPLAYFORM0 represents a sub-analysis where every word s k, i ≤ k ≤ j is a descendant of s i and where s j cannot have any other modifier (resp. can have). The two other types are defined similarly for descendants of word s j. In the first stage of the algorithm, the maximum weight of items are computed (deduced) in a bottom-up fashion. For example, the weight WEIGHT[i j] is defined as the maximum of WEIGHT DISPLAYFORM1 assumes a dependency with head s i and modifier s j. In the second stage, the algorithm retrieves arcs whose scores have contributed to the optimal objective. Part of the pseudo-code for the first and second stages are given in Algorithm 1 and Algorithm 2, respectively. Note that, usually, the second stage is implemented with a linear time complexity but we cannot rely on this optimization for our continuous relaxation. This algorithm can be thought of as the construction of a computational graph where WEIGHT, BACKPTR and CONTRIB are sets of nodes (variables). This graph includes ONE-HOT-ARGMAX operations that are not differentiable (see line 5 in Algorithm 1). This operation takes as input a vector of weights v of size k and returns a one-hot vector o of the same size with o i = 1 if and only if v i is the element of maximum value: DISPLAYFORM2 We follow a recent trend BID22 BID45 BID18 in differentiable approximation of the ONE-HOT-ARGMAX function and replace it with the PEAKED-SOFTMAX operator: DISPLAYFORM3 1≤j≤k exp(1 /τ v j) where τ > 0 is a temperature hyperparameter controlling the smoothness of the relaxation: when τ → ∞ the relaxation becomes equivalent to ONE-HOT-ARGMAX. With this update, the parsing algorithm is fully differentiable.12 Note, however, that outputs are not valid dependency trees anymore. Indeed, then an output matrix T contains continuous values that represent soft selection of arcs. BID53 introduced a alternative but similar approach for tagging with the Viterbi algorithm. We report pseudo-codes for the forward and backward passes of our continuous relaxation of EISNER's algorithm in Appendix F. The fact that T is a soft selection of arcs, and not a combinatorial structure, does not impact the decoder. Indeed, a GCN can be run over weighted graphs, the message passed between nodes is simply multiplied by the continuous weights. This is one of motivations for using GCNs rather than a Recursive LSTMs BID74 in the decoder. On the one hand, running a GCN with a matrix that represents a soft selection of arcs (i.e. with real values) has the same computational cost than using a standard adjacency matrix (i.e. with binary elements) if we use matrix multiplication on GPU. 13 On the other hand, a recursive network over a soft selection of arcs requires to build a O(n 2) set of RNN-cells that follow the dynamic programming chart where the possible inputs of a cell are multiplied by their corresponding weight in T, which is expensive and not GPU-friendly. We ran a series of experiments on 3 different languages to test our method for semi-supervised dependency parsing: English, French and Swedish. Details about corpora can be found in Appendix C. The size of each dataset is reported in TAB0. Note that the setting is especially challenging for Swedish: the amount of unlabeled data we use here barely exceeds that of labeled data. The hyperparameters of our network are described in Appendix D. In order to ensure that we do not bias our model for the benefit of the semi-supervised scenario, we use the same parameters as BID30 for the parser. Also, we did not perform any language-specific parameter selections. This makes us hope that our method can be applied to other languages with little extra effort. We stress that no part-of-speech tags are used as input in any part of our network. For English, the supervised parser took 1.5 hours to train on a NVIDIA Titan X GPU while the semi-supervised parser without sentence embedding, which sees 2 times more instances per epoch, took 3.5 hours to train. Table 2: (a) Parsing : unlabeled attachment score / labeled attachment score. We also report with the parser of BID30 which uses a different discriminative loss for supervised training. (b) Recall / Precision evaluation with respect to dependency lengths for the supervised parser and the best semi-supervised parser on the English test set. Bold numbers highlight the main differences. (c) Recall / Precision evaluation with respect to dependency labels for multi-word expressions (mwe), adverbial modifiers (advmod) and appositional modifiers (appos). For each dataset, we train under the supervised and the semi-supervised scenario. Moreover, in the semi-supervised setting, we experiment with and without latent sentence embedding z. We compare only to the model of BID30. Recently, even more accurate models have been proposed (e.g., BID12 . In principle, the ideas introduced in recent work are mostly orthogonal to our proposal as we can modify our VAE model accordingly. For example, we experimented with using bi-affine attention of BID12, though it has not turned out beneficial in our low-resource setting. Comparing to multiple previous parsers would have also required tuning each of them on our dataset, which is infeasible. Therefore, we only report with a comparable baseline, i.e. trained with a structured hinge loss BID30 BID76 . We did not perform further tuning in order to ensure that our analysis is not skewed toward one setting. Parsing are summarized in Table 2a .We observe a score increase in all three languages. Moreover, we observe that VAE performs slightly better without latent sentence embedding. We assume this is due to the fact that dependencies are more useful when no information leaks in the decoder through z. Interestingly, we observe an improvement, albeit smaller, even on Swedish, where we used a very limited amount of unlabeled data. We note that training with structured hinge loss gives stronger than our supervised baseline. In order to maintain the probabilistic interpretation of our model, we did not include a similar term in our model. We conducted qualitative analyses for English. 14 We report scores with respect to dependency lengths in Table 2b . We observe that the semi-supervised parser tends to correct two kind of errors. Firstly, it makes fewer mistakes on root attachments, i.e. the recall is similar between the two parsers but the precision of the semi-supervised one is higher. We hypothesis that root attachment errors come at a high price in the decoder because there is only a small fraction of the vocabulary that is observed with this syntactic function. Secondly, the semi-supervised parser recovers more long distance relations, i.e. the recall for dependencies with a distance superior or equal to 7 is higher. Intuitively, we assume these dependencies are more useful in the decoder: for short distance dependencies, the LSTM efficiently captures the context of the word to predict, whereas this infor-mation could be vanishing for long distances, meaning the GCN has more impact on the prediction. We also checked how the scores differ across dependency labels. We report main differences in Tables 2c. The largest improvements are obtained for multi-word expressions: this is particularly interesting because they are known to be challenging in NLP. Dependency parsing in the low-ressource scenario has been of interest in the NLP community due to the expensive nature of annotation. On the one hand, transfer approaches learn a delexicalized parser for a resource-rich language which is then used to parse a low-resource one BID2 BID52 . On the other hand, the grammar induction approach learns a dependency parser in an unsupervised manner. BID32 introduced the first generative model that outperforms the right-branching heuristic in English. Close to our work, BID6 use an auto-encoder setting where the decoder tries to rebuild the source sentence. However, their decoder is unstructured (e.g. it is not auto-regressive).Variational Auto-Encoders BID27 BID67 have been investigated in the semi-supervised settings for NLP. BID33 learn a semantic parser where the latent variable is a discrete sequence of symbols. BID85 successfully applied the variational method to semi-supervised morphological re-inflection where discrete latent variables represent linguistic features (e.g. tense, part-of-speech tag). BID82 proposed a semi-supervised semantic parser. Similarly to our model, they rely on a structured latent variable. However, all of these systems use either categorical random variables or the REINFORCE score estimator. To the best of our knowledge, no previous work used continuous relaxation of a dynamic programming latent variable in the VAE setting. The main challenge is backpropagation through discrete random variables. BID45 and BID22 first introduced the Gumbel-Softmax operator for the categorical distribution. There are two issues regarding more complex discrete distributions. Firstly, one have to build a reparametrization of the the sampling process. BID62 showed that low-order perturbations provide samples of good qualities for graphical models. Secondly, one have to build a good differentiable surrogate to the structured arg max operator. Early work replaced the structured arg max with structured attention BID26. However, computing the marginals over the parse forest is sensitive to numerical stability outside specific cases like non-projective dependency parsing BID41 BID78. BID53 proposed a stable algorithm based on dynamic program smoothing. Our approach is highly related but we describe a continuous relaxation using the parsing-as-deduction formalism. BID63 propose to replace the true gradient with a proxy that tries to satisfy constraints on a arg max operator via a projection. However, their approach is computationally expensive, so they remove the tree constraint on dependencies during backpropagation. A parallel line of work focuses on sparse structures that are differentiable BID49 BID59. We presented a novel generative learning approach for semi-supervised dependency parsing. We model the dependency structure of a sentence as a latent variable and build a VAE. We hope to motivate investigation of latent syntactic structures via differentiable dynamic programming in neural networks. Future work includes research for an informative prior for the dependency tree distribution, for example by introducing linguistic knowledge BID57 BID61 or with an adversarial training criterion BID46. This work could also be extended to the unsupervised scenario.where z is the sample. As such, e ∼ N is an input of the neural network for which we do not need to compute partial derivatives. This technique is called the reparametrization trick BID27 BID67. Sampling from a categorical distributions can be achieved through the Gumbel-Max trick BID20 BID44. Randomly generated Gumbel noise is added to the log-probability of every element of the sample space. Then, the sample is simply the element with maximum perturbed log-probability. Let d ∈ k be a random variable taking values in the corner of the unit-simplex of dimension k with probability: DISPLAYFORM0 where w is a vector of weights. Sampling d ∼ p(d) can be re-expressed as follows: DISPLAYFORM1 where G is the Gumbel distribution. Sampling g ∼ G is equivalent to setting g i = − log(− log u i)) where u i ∼ Uniform. If w is computed by a neural network, the sampling process is outside the backpropagation path. English We use the Stanford Dependency conversion BID10 of the Penn Treebank BID48 with the usual section split: 02-21 for training, 22 for development and 23 for testing. In order to simulate our framework under a low-resource setting, the annotation is kept for 10% of the training set only: a labeled sentence is the sentence which has an index (in the training set) modulo 10 equal to zero. French We use a similar setting with the French Treebank version distributed for the SPMRL 2013 shared task and the provided train/dev/test split (Abeillé et al., 2000; BID69 .Swedish We use the Talbanken dataset which contains two written text parts: the professional prose part (P) and the high school students' essays part (G). We drop the annotation of (G) in order to use this section as unlabeled data. We split the (P) section in labeled train/dev/test using a pseudo-randomized scheme. We follow the splitting scheme of but fix section 9 as development instead of k-fold cross-validation. Sentence i is allocated to section i mod 10. Then, section 1-8 are used for training, section 9 for dev and section 0 for test. Encoder: word embeddings We concatenate trainable word embeddings of size 100 with external word embeddings. 15 We use the word-dropout settings of BID30. For English, external embeddings are pre-trained with the structured skip n-gram objective BID39. 16 For French and Swedish, we use the Polyglot embeddings BID3. 17 We stress out that no part-of-speech tag is used as input in any part of our network. Encoder: dependency parser The dependency parser is built upon a two-stack BiLSTM with a hidden layer size of 125 (i.e. the output at each position is of size 250). Each dependency is then weighted using a single-layer perceptron with a tanh activation function. Arc label prediction rely on a similar setting, we refer to the reader to BID30 for more information about the parser's architecture. Encoder: sentence embedding The sentence is encoded into a fixed size vector with a simple leftto-right LSTM with an hidden size of 100. The hidden layer at the last position of the sentence is then fed to two distinct single-layer perceptrons, with an output size of 100 followed by a piecewise tanh activation function, that computes means and standard deviations of the diagonal Gaussian distribution. Decoder The decoder use fixed pre-trained embeddings only. The recurrent layer of the decoder is a LSTM with an hidden layer size of 100. MLP, MLP and MLP are all single-layer perceptrons with an output size of 100 and without activation function. Training We encourage the VAE to rely on latent structures close to the targeted ones by bootstrapping the training procedure with labeled data only. In the first two epochs, we train the network with the discriminative loss only. Then, for the next two epochs, we add the supervised ELBO term (Equation 5). Finally, after the 6th epoch, we also add the unsupervised ELBO term (Equation 3). We train our network using stochastic gradient descent for 30 epochs using Adadelta with default parameters as provided by the Dynet library. In the semisupervised scenario, we alternate between labeled and unlabeled instances. The temperature of the PEAKED-SOFTMAX operator is fixed to τ = 1. Dynamic programs for parsing have been studied as abstract algorithms that can be instantiated with different semirings BID17. For example, computing the weight of the best parse relies on the R, max, + semiring. This semiring can be augmented with set-valued operations to retrieve the best derivation. However, a straightforward implementation would have a O(n 5) space complexity: for each item in the chart, we also need to store the set of arcs. Under this formalism, the backpointer trick is a method to implicitly constructs these sets and maintain the optimal O(n 3) complexity. Our continuous relaxation replaces the max operator with a smooth surrogate and the set values with a soft-selection of sets. Unfortunately, R, PEAKED-SOFTMAX is not a commutative monoid, therefore the semiring analogy is not transposable. We describe how we can embed a continuous relaxation of projective dependency parsing as a node in a neural network. During the forward pass, we are given arc weights W and we compute the relaxed projective dependency tree T that maximize the arc-factored weight h,m T h,m ×W h,m. Each output variable T h,m ∈ is a soft selection of dependency with head-word s h and modifier s m. During back-propagation, we are given partial derivatives of the loss with respect to each arc and we compute the ones with respect to arc weights: DISPLAYFORM0 Note that the Jacobian matrix has O(n 4) values but we do need to explicitly compute it. The space and time complexity of the forward and backward passes are both cubic, similar to Eisner's algorithm. The forward pass is a two step algorithm:1. First, we compute the cumulative weight of each item and store soft backpointers to keep track of contribution of antecedents. This step is commonly called to inside algorithm. 2. Then, we compute the contribution of each arc thanks to the backpointers. This step is somewhat similar to the arg max reconstruction algorithm. The outline of the algorithm is given in Algorithm 3.The inside algorithm computes the following variables:• a[i j][k] is the weight of item [i j] if we split its antecedent at k.• b[i j][k] is the soft backpointer to antecedents of item [i j] with split at k.• c[i j] is the cumulative weight of item [i j].and similarly for the other chart values. The algorithm is given in Algorithm 5.The backpointer reconstruction algorithm compute the contribution of each arc. We follow backpointers in reverse order in order to compute the contribution of each itemc[i j]. The algorithm is given in Algorithm 6. During the backward pass, we compute the partial derivatives of variables using the chain rule, i.e. in the reverse order of their creation: we first run backpropagation through the backpointer reconstruction algorithm and then through the inside algorithm (see Algorithm 4). Given the partial derivatives in FIG3, backpropagation through the backpointer reconstruction algorithm is straighforward to compute, see Algorithm 7. Partial derivatives of the inside algorithm's variables are given in Figure 4. ∀i < k ≤ j: DISPLAYFORM0

Represent the TLDR for finding the corresponding scientific article

Differentiable dynamic programming over perturbed input weights with application to semi-supervised VAE

Represent the short summary for finding the corresponding scientific article (long)

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.


Question: Represent the text to find a very short summary (1-2 sentences max):

Question: DeConvNet, Guided BackProp, LRP, were invented to better understand deep neural networks. We show that these methods do not produce the theoretically correct explanation for a linear model. Yet they are used on multi-layer networks with millions of parameters. This is a cause for concern since linear models are simple neural networks. We argue that explanation methods for neural nets should work reliably in the limit of simplicity, the linear models. Based on our analysis of linear models we propose a generalization that yields two explanation techniques (PatternNet and PatternAttribution) that are theoretically sound for linear models and produce improved explanations for deep networks. Deep learning made a huge impact on a wide variety of applications BID8 BID24 BID9 BID15 BID10 BID18 and recent neural network classifiers have become excellent at detecting relevant signals (e.g., the presence of a cat) contained in input data points such as images by filtering out all other, nonrelevant and distracting components also present in the data. This separation of signal and distractors is achieved by passing the input through many layers with millions of parameters and nonlinear activation functions in between, until finally at the output layer, these models yield a highly condensed version of the signal, e.g. a single number indicating the probability of a cat being in the image. While deep neural networks learn efficient and powerful representations, they are often considered a'black-box'. In order to better understand classifier decisions and to gain insight into how these models operate, a variety techniques have been proposed BID20 BID25 BID12 BID1 BID0 BID11 BID26 BID22 BID27 BID23 BID21. These methods for explaining classifier decisions operate under the assumption that it is possible to propagate the condensed output signal back through the classifier to arrive at something that shows how the relevant signal was encoded in the input and thereby explains the classifier decision. Simply put, if the classifier detected a cat, the visualization should point to the cat-relevant aspects of the input image from the perspective of the network. Techniques that are based on this principle include saliency maps from network gradients BID1 BID20, DeConvNet (, DCN), Guided BackProp (, GBP), Figure 1: Illustration of explanation approaches. Function and signal approximators visualize the explanation using the original color channels. The attribution is visualized as a heat map of pixelwise contributions to the output Layer-wise Relevance Propagation (, LRP) and the Deep Taylor Decomposition (, DTD), Integrated Gradients BID23 and SmoothGrad BID21.The merit of explanation methods is often demonstrated by applying them to state-of-the-art deep learning models in the context of high dimensional real world data, such as ImageNet, where the provided explanation is intuitive to humans. Unfortunately, theoretical analysis as well as quantitative empirical evaluations of these methods are lacking. Deep neural networks are essentially a composition of linear transformations connected with nonlinear activation functions. Since approaches, such as DeConvNet, Guided BackProp, and LRP, back-propagate the explanations in a layer-wise fashion, it is crucial that the individual linear layers are handled correctly. In this work we show that these gradient-based methods fail to recover the signal even for a single-layer architecture, i.e. a linear model. We argue that therefore they cannot be expected to reliably explain a deep neural network and demonstrate this with quantitative and qualitative experiments. In particular, we provide the following key contributions:• We analyze the performance of existing explanation approaches in the controlled setting of a linear model (Sections 2 and 3).• We categorize explanation methods into three groups -functions, signals and attribution (see Fig. 1) -that require fundamentally different interpretations and are complementary in terms of information about the neural network (Section 3).• We propose two novel explanation methods -PatternNet and PatternAttribution -that alleviate shortcomings of current approaches, as discovered during our analysis, and improve explanations in real-world deep neural networks visually and quantitatively (Sections 4 and 5).This presents a step towards a thorough analysis of explanation methods and suggests qualitatively and measurably improved explanations. These are crucial requirements for reliable explanation techniques, in particular in domains, where explanations are not necessarily intuitive, e.g. in health and the sciences BID16.Notation and scope Scalars are lowercase letters (i), column vectors are bold (u), element-wise multiplication is . The covariance between u and v is cov [u, v], the covariance of u and i is cov [u, i]. The variance of a scalar random variable i is σ 2 i. Estimates of random variables will have a hat (û). We analyze neural networks excluding the final soft-max output layer. To allow for analytical treatment, we only consider networks with linear neurons optionally followed by a rectified linear unit (ReLU), max-pooling or soft-max. We analyze linear neurons and nonlinearities independently such that every neuron has its own weight vector. These restrictions are similar to those in the saliency map BID20, DCN , GBP (Springenberg Figure 2 : For linear models, i.e., a simple neural network, the weight vector does not explain the signal it detects BID5 . The data x = ya s + a d is color-coded w.r.t. the output y = w T x. Only the signal s = ya s contributes to y. The weight vector w does not agree with the signal direction, since its primary objective is canceling the distractor. Therefore, rotations of the basis vector a d of the distractor with constant signal s lead to rotations of the weight vector (right). et al., 2015), LRP BID0 and DTD BID11. Without loss of generality, biases are considered constant neurons to enhance clarity. In this section, we analyze explanation methods for deep neural network, starting with the simplest neural network setting: a purely linear model and data sampled from a linear generative model. This setup allows us to (i) fully control how signal and distractor components are encoded in the input data and (ii) analytically track how the ing explanation relates to the known signal component. This analysis allows us then to highlight shortcomings of current explanation approaches that carry over to deep neural networks. Consider the following toy example (see Fig. 2) where we generate data x as: DISPLAYFORM0 We train a linear regression model to extract y from x. By construction, s is the signal in our data, i.e., the part of x containing information about y. Using the terminology of BID5 the distractor d obfuscates the signal making the detection task more difficult. To optimally extract y, our model has to be able to filter out the distractor d. This is why the weight vector is also called the filter. In the example, w = [1, −1] T fulfills this convex task. From this example, we can make several observations: The optimal weight vector w does not align, in general, with the signal direction a s, but tries to filter the contribution of the distractor (see Fig. 2). This is optimally solved when the weight vector is orthogonal to the distractor w T d = 0. Therefore, when the direction of the distractor a d changes, w must follow, as illustrated on the right hand side of the figure. On the other hand, a change in signal direction a s can be compensated for by a change in sign and magnitude of w such that w T a s = 1, but the direction stays constant. The fact that the direction of the weight vector in a linear model is largely determined by the distractor implies that given only the weight vector, we cannot know what part of the input produces the output y. On the contrary, the direction a s must be learned from data. Now assume that we have additive isotropic Gaussian noise. The mean of the noise can easily be compensated for with a bias change. Therefore, we only have to consider the zero-mean case. Since isotropic Gaussian noise does not contain any correlations or structure, the only way to remove it is by averaging over different measurements. It is not possible to cancel it out effectively by using a well-chosen weight vector. However, it is well known that adding Gaussian noise shrinks the weight vector and corresponds to L2 regularization. In the absence of a structured distractor, the smallest weight vector w such that w T a s = 1 is the one in the direction of the signal. Therefore in practice both these effects influence the actual weight vector. As already indicated above, deep neural networks are essentially a composition of linear layers and non-linear activation functions. In the next section, we will show that gradient-based methods, e.g., DeConvNet, Guided BackProp, and LRP, are not able to distinguish signal from distractor in a linear model and therefore back-propagate sub-optimal explanations in deeper networks. This analysis allows us to develop improved layer-wise explanation techniques and to demonstrate quantitative and qualitative better explanations for deep neural networks. Terminology Throughout this manuscript we will use the following terminology: The filter w tells us how to extract the output y optimally from data x. The pattern a s is the direction in the data along which the desired output y varies. Both constitute the signal s = a s y, i.e., the contributing part of x. The distractor d is the component of the data that does not contain information about the desired output. In this section, we take a look at a subset of explanation methods for individual classifier decisions and discuss how they are connected to our analysis of linear models in the previous section. Fig. 1 gives an overview of the different types of explanation methods which can be divided into function, signal and attribution visualizations. These three groups all present different information about the network and complement each other. Functions -gradients, saliency map Explaining the function in input space corresponds to describing the operations the model uses to extract y from x. Since deep neural networks are highly nonlinear, this can only be approximated. The saliency map estimates how moving along a particular direction in input space influences y (i.e., sensitivity analysis) where the direction is given by the model gradient BID1 BID20. In case of a linear model y = w T x, the saliency map reduces to analyzing the weights ∂y/∂x = w. Since it is mostly determined by the distractor, as demonstrated above, it is not representing the signal. It tells us how to extract the signal, not what the signal is in a deep neural network. Signal -DeConvNet, Guided BackProp, PatternNet The signal s detected by the neural network is the component of the data that caused the networks activations. BID26 formulated the goal of these methods as " [...] to map these activities back to the input pixel space, showing what input pattern originally caused a given activation in the feature maps". In a linear model, the signal corresponds to s = a s y. The pattern a s contains the signal direction, i.e., it tells us where a change of the output variable is expected to be measurable in the input BID5. Attempts to visualize the signal for deep neural networks were made using DeConvNet BID26 and Guided BackProp BID22. These use the same algorithm as the saliency map, but treat the rectifiers differently (see Fig. 1): DeConvNet leaves out the rectifiers from the forward pass, but adds additional ReLUs after each deconvolution, while Guided BackProp uses the ReLUs from the forward pass as well as additional ones. The back-projections for the linear components of the network correspond to a superposition of what are assumed to be the signal directions of each neuron. For this reason, these projections must be seen as an approximation of the features that activated the higher layer neuron. It is not a reconstruction in input space BID26.For the simplest of neural networks -the linear model -these visualizations reduce to the gradient 1. They show the filter w and neither the pattern a s, nor the signal s. Hence, DeConvNet and Guided BackProp do not guarantee to produce the detected signal for a linear model, which is proven by our toy example in Fig. 2. Since they do produce compelling visualizations, we will later investigate whether the direction of the filter w coincides with the direction of the signal s. We will show that this is not the case and propose a new approach, PatternNet (see Fig. 1), to estimate the correct direction that improves upon the DeConvNet and Guided BackProp visualizations. Attribution -LRP, Deep Taylor Decomposition, PatternAttribution Finally, we can look at how much the signal dimensions contribute to the output through the layers. This will be referred to as the attribution. For a linear model, the optimal attribution would be obtained by element-wise multiplying the signal with the weight vector: r input = w ay, with the element-wise multiplication. BID0 introduced layer-wise relevance propagation (LRP) as a decomposition of pixel-wise contributions (called relevances). BID11 extended this idea and proposed the deep Taylor decomposition (DTD). The key idea of DTD is to decompose the activation of a neuron in terms of contributions from its inputs. This is achieved using a first-order Taylor expansion around a root point x 0 with w T x 0 = 0. The relevance of the selected output neuron i is initialized with its output from the forward pass. The relevance from neuron i in layer l is re-distributed towards its input as: DISPLAYFORM0 To obtain the relevance for neuron i in layer l−1 the incoming relevances from all connected neurons j in layer l are summed r DISPLAYFORM1 Here we can safely assume that w T x > 0 because a non-active ReLU unit from the forward pass stops the re-distribution in the backward pass. This is identical to how a ReLU stops the propagation of the gradient. The difficulty in the application of the deep Taylor decomposition is the choice of the root point x 0, for which many options are available. It is important to recognize at this point that selecting a root point for the DTD corresponds to estimating the distractor x 0 = d and, by that, the signalŝ = x − x 0. PatternAttribution is a DTD extension that learns from data how to set the root point. Summarizing, the function extracts the signal from the data by removing the distractor. The attribution of output values to input dimensions shows how much an individual component of the signal contributes to the output, which is what LRP calls relevance. Visualizing the function has proven to be straightforward BID1 BID20. In contrast, visualizing the signal BID5 BID26 BID22 and the attribution BID0 BID11 BID23 is more difficult. It requires a good estimate of what is the signal and what is the distractor. In the following section we first propose a quality measure for neuron-wise signal estimators. This allows us to evaluate existing approaches and, finally, derive signal estimators that optimize this criterion. These estimators will then be used to explain the signal (PatternNet) and the attribution (PatternAttribution). All mentioned techniques as well as our proposed signal estimators treat neurons independently, i.e., the full explanation will be a superposition of neuron-wise explanations. Recall that the input data x comprises both signal and distractor: x = s + d, and that the signal contributes to the output but the distractor does not. Assuming the filter w has been trained sufficiently well to extract y, we have DISPLAYFORM0 Note that estimating the signal based on these conditions alone is an ill-posed problem. We could limit ourselves to linear estimators of the formŝ = u(w T u) −1 y, with u a random vector such that DISPLAYFORM1 For such an estimator, the signal estimateŝ = u w T u −1 y satisfies w Tŝ = y. This implies the existence of an infinite number of possible rules for the DTD as well as infinitely many back-projections for the DeConvNet family. To alleviate this issue, we introduce the following quality measure ρ for a signal estimator S(x) =ŝ that will be written with explicit variances and covariances using the shorthandsd = x − S(x) and y = w T x: DISPLAYFORM2 This criterion introduces an additional constraint by measuring how much information about y can be reconstructed from the residuals x −ŝ using a linear projection. The best signal estimators remove most of the information in the residuals and thus yield large ρ(S). Since the correlation is invariant to scaling, we constrain v Td to have variance σ 2 v Td = σ 2 y. Finding the optimal v for a fixed S(x) amounts to a least-squares regression fromd to y. This enables us to assess the quality of signal estimators efficiently. Let us now discuss two signal estimators that have been used in previous approaches. S x -the identity estimator The naive approach to signal estimation is to assume the entire data is signal and there are no distractors: DISPLAYFORM0 With this being plugged into the deep Taylor framework, we obtain the z-rule BID11 which is equivalent to LRP BID0. For a linear model, this corresponds to r = w x as the attribution. It can be shown that for ReLU and max-pooling networks, the z-rule reduces to the element-wise multiplication of the input and the saliency map BID17 BID6. This means that for a whole network, the assumed signal is simply the original input image. It also implies that, if there are distractors present in the data, they are included in the attribution: r = w x = w s + w d. When moving through the layers by applying the filters w during the forward pass, the contributions from the distractor d are cancelled out. However, they cannot be cancelled in the backward pass by the element-wise multiplication. The distractor contributions w d that are included in the LRP explanation cause the noisy nature of the visualizations based on the z-rule. S w -the filter based estimator The implicit assumption made by DeConvNet and Guided BackProp is that the detected signal varies in the direction of the weight vector w. This weight vector has to be normalized in order to be a valid signal estimator. In the deep Taylor decomposition framework this corresponds to the w 2 -rule and in the following signal estimator: DISPLAYFORM1 For a linear model, this produces an attribution of the form w w w T w y. This estimator does not reconstruct the proper signal in the toy example of section 2. Empirically it is also sub-optimal in our experiment in Fig. 3. We suggest to learn the signal estimator S from data by optimizing the previously established criterion. A signal estimator S is optimal with respect to Eq. if the correlation is zero for all possible v: ∀v, cov[y,d]v = 0. This is the case when there is no covariance between y andd. Because of linearity of the covariance and sinced = x − S(x) the above condition leads to DISPLAYFORM0 It is important to recognize that the covariance is a summarizing statistic and consequently the problem can still be solved in multiple ways. We will present two possible solutions to this problem. Note that when optimizing the estimator, the contribution from the bias neuron will be considered 0 since it does not covary with the output y. S a -The linear estimator A linear neuron can only extract linear signals s from its input x. Therefore, we could assume a linear dependency between s and y, yielding a signal estimator: DISPLAYFORM1 Plugging this into Eq. and optimising for a yields DISPLAYFORM2 Note that this solution is equivalent to the approach commonly used in neuro-imaging BID5 despite different derivation. With this approach we can recover the signal of our toy example in section 2. It is equivalent to the filter-based approach only if the distractors are orthogonal to the signal. We found that the linear estimator works well for the convolutional layers. However, when using this signal estimator with ReLUs in the dense layers, there is still a considerable correlation left in the distractor component (see Fig. 3).S a+− -The two-component estimator To move beyond the linear signal estimator, it is crucial to understand how the rectifier influences the training. Since the gate of the ReLU closes for negative activations, the weights only need to filter the distractor component of neurons with y > 0. Since this allows the neural network to apply filters locally, we cannot assume a global distractor component. We rather need to distinguish between the positive and negative regime: DISPLAYFORM3 Even though signal and distractor of the negative regime are canceled by the following ReLU, we still need to make this distinction in order to approximate the signal. Otherwise, information about whether a neuron fired would be retained in the distractor. Thus, we propose the two-component signal estimator: DISPLAYFORM4 Next, we derive expressions for the patterns a + and a −. We denote expectations over x within the positive and negative regime with E + [x] and E − [x], respectively. Let π + be the expected ratio of inputs x with w T x > 0. The covariance of data/signal and output become: DISPLAYFORM5 Assuming both covariances are equal, we can treat the positive and negative regime separately using Eq. to optimize the signal estimator: FORMULA12 and solving for a + yields the required parameter (a − analogous). DISPLAYFORM6 DISPLAYFORM7 The solution for S a+− reduces to the linear estimator when the relation between input and output is linear. Therefore, it solves our introductory linear example correctly. Based on the presented analysis, we propose PatternNet and PatternAttribution as illustrated in Fig. 1. PatternNet yields a layer-wise back-projection of the estimated signal to input space. The signal estimator is approximated as a superposition of neuron-wise, nonlinear signal estimators S a+− in each layer. It is equal to the computation of the gradient where during the backward pass the weights of the network are replaced by the informative directions. In Fig. 1, a visual improvement over DeConvNet and Guided Backprop is apparent. PatternAttribution exposes the attribution w a + and improves upon the layer-wise relevance propagation (LRP) framework BID0. It can be seen as a root point estimator for the DeepTaylor Decomposition (DTD). Here, the explanation consists of neuron-wise contributions of the estimated signal to the classification score. By ignoring the distractor, PatternAttribution can reduce the noise and produces much clearer heat maps. By working out the back-projection steps in the Deep-Taylor Decomposition with the proposed root point selection method, it becomes obvious that PatternAttribution is also analogous to the backpropagation operation. In this case, the weights are replaced during the backward pass by w a +. To evaluate the quality of the explanations, we focus on the task of image classification. Nevertheless, our method is not restricted to networks operating on image inputs. We used Theano BID2 and BID4 for our implementation. We restrict the analysis to the well-known ImageNet dataset BID13 using the pre-trained VGG-16 model BID19. Images were rescaled and cropped to 224x224 pixels. The signal estimators are trained on the first half of the training dataset. The vector v, used to measure the quality of the signal estimator ρ(x) in Eq. FORMULA5, is optimized on the second half of the training dataset. This enables us to test the signal estimators for generalization. All the presented here were obtained using the official validation set of 50000 samples. The validation set was not used for training the signal estimators, nor for training the vector v to measure the quality. Consequently our are obtained on previously unseen data. The linear and the two component signal estimators are obtained by solving their respective closed form solutions (Eq. and Eq. FORMULA15 ). With a highly parallelized implementation using 4 GPUs this could be done in 3-4 hours. This can be considered reasonable given that several days are required to train the actual network. The quality of a signal estimator is assessed with Eq.. Solving it with the closed form solution is computationally prohibitive since it must be repeated for every single weight vector in the network. Therefore we optimize the equivalent least-squares problem using stochastic mini-batch gradient descent with until convergence. This was implemented on a NVIDIA Tesla K40 and took about 24 hours per optimized signal estimator. After learning to explain, individual explanations are computationally cheap since they can be implemented as a back-propagation pass with a modified weight vector. As a , our method produces explanations at least as fast as the work by BID3 on real time saliency. However, our method has the advantage that it is not only applicable to image models but is a generalization of the theory commonly used in neuroimaging BID5. Measuring the quality of signal estimators In Fig. 3 we present the from the correlation measure ρ(x), where higher values are better. We use random directions as baseline signal estimators. Clearly, this approach removes almost no correlation. The filter-based estimator S w succeeds in removing some of the information in the first layer. This indicates that the filters are similar to the patterns in this layer. However, the gradient removes much less information in the higher layers. Overall, it does not perform much better than the random estimator. This implies that the weights do not correspond to the detected stimulus in a neural network. Hence the implicit assumptions about the signal made by DeConvNet and Guided BackProp is not valid. The optimized estimators remove much more of the correlations across the board. For convolutional layers, S a and S a+− perform comparably in all but one layer. The two component estimator S a+− is best in the dense layers. Image degradation The first experiment was a direct measurement of the quality of the signal estimators of individual neurons. The second one is an indirect measurement of the quality, but it considers the whole network. We measure how the prediction (after the soft-max) for the initially selected class changes as a function of corrupting more and more patches based on the ordering assigned by the attribution (see BID14 . This is also related to the work by BID27 . In this experiment, we split the image in non-overlapping patches of 9x9 pixels. We compute the attribution and sum all the values within a patch. We sort the patches in decreasing order based on the aggregate heat map value. In step n = 1..100 we replace the first n patches with the their mean per color channel to remove the information in this patch. Then, we measure how this influences the classifiers output. We use the estimators from the previous experiment to obtain the function-signal attribution heat maps for evaluation. A steeper decay indicates a better heat map. Results are shown in Fig. 4 . The baseline, in which the patches are randomly ordered, performs worst. The linear optimized estimator S a performs quite poorly, followed by the filter-based estimator S w . The trivial signal estimator S x performs just slightly better. However, the two component model S a+− leads to the fastest decrease in confidence in the original prediction by a large margin. Its excellent quantitative performance is also backed up by the visualizations discussed next. Qualitative evaluation In FIG1, we compare all signal estimators on a single input image. For the trivial estimator S x, the signal is by definition the original input image and, thus, includes the distractor. Therefore, its noisy attribution heat map shows contributions that cancel each other in the neural network. The S w estimator captures some of the structure. The optimized estimator S a in slightly more structure but struggles on color information and produces dense heat maps. The two component model S a+− on the right captures the original input during signal estimation and produces a crisp heat map of the attribution. FIG2 shows the visualizations for six randomly selected images from ImageNet. PatternNet is able to recover a signal close to the original without having to resort to the inclusion of additional rectifiers in contrast to DeConvNet and Guided BackProp. We argue that this is due to the fact that the optimization of the pattern allows for capturing the important directions in input space. This contrasts with the commonly used methods DeConvNet, Guided BackProp, LRP and DTD, for which the correlation experiment indicates that their implicit signal estimator cannot capture the true signal in the data. Overall, the proposed approach produces the most crisp visualization in addition to being measurably better, as shown in the previous section. Additonally, we also contrast our methods to the prediction-differences analysis by BID27 in the supplementary material. Relation to previous methods Our method can be thought of as a generalization of the work by BID5, making it applicable on deep neural networks. Remarkably, our proposed approach can solve the toy example in section 2 optimally while none of the previously published methods for deep learning are able to solve this BID0 BID11 BID21 BID23 BID27 BID3 BID26 BID22 . Our method shares the idea that to explain a model properly one has to learn how to explain it with Zintgraf et al. FORMULA5 and BID3 . Furthermore, since our approach is after training just as expensive as a single back-propagation step, it can be applied in a real-time context, which is also possible for the work done by BID3 but not for BID27 . Understanding and explaining nonlinear methods is an important challenge in machine learning. Algorithms for visualizing nonlinear models have emerged but theoretical contributions are scarce. We have shown that the direction of the model gradient does not necessarily provide an estimate for the signal in the data. Instead it reflects the relation between the signal direction and the distracting noise contributions ( Fig. 2). This implies that popular explanation approaches for neural networks (DeConvNet, Guided BackProp, LRP) do not provide the correct explanation, even for a simple linear model. Our reasoning can be extended to nonlinear models. We have proposed an objective function for neuron-wise explanations. This can be optimized to correct the signal visualizations (PatternNet) and the decomposition methods (PatternAttribution) by taking the data distribution into account. We have demonstrated that our methods constitute a theoretical, qualitative and quantitative improvement towards understanding deep neural networks. This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement NO 657679, the BMBF for the Berlin Big Data Center BBDC (01IS14013A), a hardware donation from NVIDIA. We thank Sander Dieleman, Jonas Degraeve, Ira Korshunova, Stefan Chmiela, Malte Esders, Sarah Hooker, Vincent Vanhoucke for their comments to improve this manuscript. We are grateful to Chris Olah and Gregoire Montavon for the valuable discussions. In this section we will give an overview of the visualization algorithms to clarify their actual implementation for ReLu networks. This shows the similarities and the differences between all approaches. For all visualization approaches, the back-projection through a max-pooling layer is only through the path that was active in the forward pass. To create the Predictive-Differences analysis visualizations BID27, we used the opensource code provided by the authors with the default parameter settings provided for VGG. Figure 7: Visualization of random images from ImageNet (validation set). In the leftmost shows column the ground truth, the predicted label and the classifier's confidence. Comparison between the proposed methods PatternNet and PatternAttribution to the Prediction-Differences approach by BID27.

Without learning, it is impossible to explain a machine learning model's decisions.

Represent the TLDR for finding the corresponding scientific article

We propose a extension of the batch normalization, show a first-of-its-kind convergence analysis for this extension and show in numerical experiments that it has better performance than the original batch normalizatin.