Question: represent the scientific article to find a short tldr

Question: Neural network models have shown excellent fluency and performance when applied to abstractive summarization. Many approaches to neural abstractive summarization involve the introduction of significant inductive bias, such as pointer-generator architectures, coverage, and partially extractive procedures, designed to mimic human summarization. We show that it is possible to attain competitive performance by instead directly viewing summarization as language modeling. We introduce a simple procedure built upon pre-trained decoder-transformers to obtain competitive ROUGE scores using a language modeling loss alone, with no beam-search or other decoding-time optimization, and instead rely on efficient nucleus sampling and greedy decoding. Neural network approaches to abstractive summarization generally encode the source document into some hidden state or representation, then decode this representation into a summarized, abstracted version of the source document. These approaches usually rely on a sequence-to-sequence style architecture, and tend to produce fluent, well formed natural language summaries when coupled with beam search or other decoding techniques. A weakness of traditional sequence-to-sequence learning when applied to summarization is the lack of a direct copy mechanism, leading to missing or misrepresented details in decoded summaries. Though attention helps ameliorate this issue by directly learning to focus on specific words or phrases in a source document, many have allowed for an explicit copy mechanism inspired by Pointer Networks, by optimizing a differentiable decision whether to generate new text or directly copy from the source. Peters et al., Devlin et al., Radford et al., and many others have shown the benefits of large-scale pretraining on large, unlabeled corpora on a variety of downstream tasks in transfer learning settings. In particular, it has been shown that large-scale, attention-only language modeling via decoder-only transformers as an unsupervised pretraining task admits the ability to perform zero-shot learning on meaningful tasks involving natural language generation. Motivated by this, we propose a simple method that exhibits competitive performance on abstractive summarization without using sequence-to-sequence architectures or other standard tools in the neural abstractive summarization toolbox, and instead using a decoder-only transformer language model with transfer learning. This further illustrates the utility of finetuning language models trained on open domain text. Transformer Preliminaries Our model builds on previous work utilizing decoder-only Transformers for jointly learning language modeling and sequence transduction in aligned domains, which limits attention to tokens 0, 1,..., n − 1 for predicting token n. Formally, a decoder-only Transformer considers a sequence of one-hot token vectors T = [t 0, t 1, . . ., t n−1] ∈ {0, 1} V ×n, with each t i ∈ {0, 1} V where V is the size of the vocabulary. Given an embedding matrix W E ∈ R d×V and a positional encoding matrix W P ∈ R d×(n−1), the model computes where TRF is the transformer block with self-attention, first introduced in Vaswani et al.. We utilize the modifications provided in Radford et al., such as moving Layer Normalization to the beginning of each transformer block. Decoder-only Sequence Transduction for Summarization Formally, consider a set of paired documents C = {(x, y)}, |C| = N. For a source-summary pair (x, y) ∈ C, the source document x = [x 0, . . ., x m] and reference summary y = [y 0, . . ., y k] are sequences of one-hot token vectors. To learn this mapping using a language model, we combine x and y using special learnable vectors corresponding to control tokens. In addition, we augment Eq. 1 to include a segment-specific (i.e., source or summary) embedding. Finally, we reset the positional encoding for the summary. Our model is fed three sequences (see Eq. 2): a concatenation of the source document and the summary (S), positional encodings that reset for the summary component (P), and segment-specific encodings for the source and the summary (Q). We represent the start of the source document with α, the beginning of the summary with β, and the end of sequence with δ. Additionally, we encode the source segment with σ and the summary segment with τ. Thus, our model changes Eq. 1 by adding the position encoding modification from Eq. 2 and an additional trainable weight W Q representing the segment encoding Q. The model is trained via maximum likelihood, where we take into account the full likelihood of the source-summary pair. Input Representation Given recent trends moving away from purely word-or character-level representations, we utilize data-driven subword encoding via Byte Pair Encoding (BPE), following the procedure outlined in Radford et al.. For experiments in which we finetune the 117M parameter model from Radford et al., we utilize their prebuilt vocabulary; in ablation studies, we utilize SentencePiece to learn BPE merges. Datasets We train and evaluate our models on the CNN/Daily Mail (CNN-DM) corpus of news articles and summaries, utilizing the non-anonymized version. We use the predefined training, validation, and test splits, and limit source articles to 400 tokens and summaries to 100 tokens at training time. As an additional test, we train and evaluate the best model configuration from the ablation studies above on the Extreme Summarization (XSum) corpus, which contains single sentence summaries of BBC articles. As shown in Narayan et al., the XSum corpus requires models to perform a much higher degree of semantic distillation, as indicated by low n-gram overlap, high n-gram novelty, and poorly performing LEAD-3 baselines. We conduct experiments in two regimes for CNN-DM: first, we finetune the model outlined in Sec. 2 on top of the 117M parameter model release from Radford et al., and second, we perform a full training from scratch in order to ablate the effect of transfer learning. We utilize a context size of 1024 with an embedding dimension of 768, 12 attention heads, and a batch size of 10. We train using the Adam optimizer with a learning rate of 5 × 10 −5 until the loss ceases to decrease on the validation set. For XSum, we use the highest-performing setup from CNN-DM experiments. In lieu of beam search, we compare greedy decoding and nucleus sampling. In both cases, we decode until we reach the stop-token δ (Eq. 2). In the case of nucleus sampling, we perform 5 Table 1: Comparison of our method with select existing methods on the CNN-DM dataset. independent decodings 1 with p = 0.3, then pick the decoding that reports the lowest negative log likelihood score of the completed summary. We use 1/k 0.6 as a likelihood normalization term to avoid a preference for shorter summaries, borrowing directly from Wu et al.. Table 2: Ablation of model components on CNN-DM (Decoded via nucleus sampling procedure). We evaluate all models using the ROUGE metric, in particular the F1 variants of ROUGE-1, ROUGE-2, and ROUGE-L which measure unigram overlap, bigram overlap, and longest common subsequence respectively. CNN-DM Our main are displayed in Table 1, where we compare our method (in the bottom section of the table) to existing methods (in the upper portion) on the CNN-DM dataset, and show ablations in Table 2. We note that our models (for ROUGE-1 and -2) are competitive even when using greedy decoding, and without any sequence-to-sequence style architectures or coverage terms, illustrating the power of this approach for abstractive summarization. We note that using a well trained language model and then finetuning yields a significant performance jump (as shown via ablation in Table 2), motivating this method in practical contexts given the recent trends toward large-scale, self-supervised learning approaches. XSum As a secondary evaluation of our approach, we train our best model on the XSum dataset and report ROUGE scores in a direct comparison to the benchmarks reported. Results for these experiments are shown in Table 3. We achieve highly competitive performance relative to models reported in Narayan et al. building on a finetuning approach without using many of the inductive biases traditionally present in summarization methods. This work puts forward a simple approach to abstractive summarization by viewing sequence transduction as a language modeling problem. We show the effectiveness of using decoder-only transformers for this task, in particular, when coupled with recent advances in large-scale language modeling and transfer learning. We show that competitive performance on two benchmark datasets is possible without many of the standard tools in neural abstractive summarization, such as sequence-tosequence modeling, coverage mechanisms, direct ROUGE optimization via reinforcement learning, or beam search, instead relying on a purely language modeling loss and simple decoding mechanisms such as nucleus sampling and greedy decoding. This approach yields highly fluent text, and illustrates the power of unsupervised representation learning-based transfer learning for downstream tasks. Table 3: Comparison of with existing methods on XSum, reported in Narayan et al..

represent the tldr for finding the corresponding scientific article

Example:
Provided: "We constructively prove that even the slightest nonlinear activation functions introduce spurious local minima, for general datasets and activation functions." Match: "We investigate the loss surface of neural networks. We prove that even for one-hidden-layer networks with "slightest" nonlinearity, the empirical risks have spurious local minima in most cases. Our thus indicate that in general "no spurious local minim" is a property limited to deep linear networks, and insights obtained from linear networks may not be robust. Specifically, for ReLU(-like) networks we constructively prove that for almost all practical datasets there exist infinitely many local minima. We also present a counterexample for more general activations (sigmoid, tanh, arctan, ReLU, etc.), for which there exists a bad local minimum. Our make the least restrictive assumptions relative to existing on spurious local optima in neural networks. We complete our discussion by presenting a comprehensive characterization of global optimality for deep linear networks, which unifies other on this topic. Neural network training reduces to solving nonconvex empirical risk minimization problems, a task that is in general intractable. But success stories of deep learning suggest that local minima of the empirical risk could be close to global minima. BID5 use spherical spin-glass models from statistical physics to justify how the size of neural networks may in local minima that are close to global. However, due to the complexities introduced by nonlinearity, a rigorous understanding of optimality in deep neural networks remains elusive. Initial steps towards understanding optimality have focused on deep linear networks. This area has seen substantial recent progress. In deep linear networks there is no nonlinear activation; the output is simply a multilinear function of the input. BID1 prove that some shallow networks have no spurious local minima, and extends this to squared error deep linear networks, showing that they only have global minima and saddle points. Several other works on linear nets have also appeared (; ; ; ; a; b).The theory of nonlinear neural networks (which is the actual setting of interest), however, is still in its infancy. There have been attempts to extend the "local minima are global" property from linear to nonlinear networks, but recent suggest that this property does not usually hold . Although not unexpected, rigorously proving such turns out to be non-trivial, forcing several authors (e.g., ; BID8 ;) to make somewhat unrealistic assumptions (realizability and Gaussianity) on data. In contrast, we prove existence of spurious local minima under the least restrictive (to our knowledge) assumptions. Since seemingly subtle changes to assumptions can greatly influence the analysis as well as the applicability of known , let us first summarize what is known; this will also help provide a better intuitive perspective on our (as the technical details are somewhat involved). There is a large and rapidly expanding literature of optimization of neural networks. Some works focus on the loss surface BID1;;;;;; Safran & Shamir, 1.2 CONTRIBUTIONS AND SUMMARY OF We summarize our key contributions more precisely below. Our work encompasses for both nonlinear and linear neural networks. First, we study whether the "local minima are global" property holds for nonlinear networks. Unfortunately, our here are negative. Specifically, we prove For piecewise linear and nonnegative homogeneous activation functions (e.g., ReLU), we prove in Theorem 1 that if linear models cannot perfectly fit the data, one can construct infinitely many local minima that are not global. In practice, most datasets are not linearly fittable, hence this gives a constructive proof of spurious local minima for generic datasets. In contrast, several existing either provide only one counterexample , or make restrictive assumptions of realizability (; BID8 or linear separability (a). This is presented in Section 2.In Theorem 2 we tackle more general nonlinear activation functions, and provide a simple architecture (with squared loss) and dataset, for which there exists a local minimum inferior to the global minimum for a realizable dataset. Our analysis applies to a wide range of activations, including sigmoid, tanh, arctan, ELU BID6, SELU , and ReLU. Considering that realizability of data simplifies the analysis and ensures zero loss at global optima, our counterexample that is realizable and yet has a spurious local minimum is surprising, suggesting that the situation is likely worse for non-realizable data. See Section 3 for details. We complement our negative by presenting the following positive on linear networks: Assume that the hidden layers are as wide as either the input or the output, and that the empirical risk ((W j) H+1 j=1 ) equals 0 (W H+1 W H · · · W 1), where 0 is a differentiable loss function and W i is the weight matrix for layer i. Theorem 4 shows if (Ŵ j) H+1 j=1 is a critical point of, then its type of stationarity (local min/max, or saddle) is closely related to the behavior of 0 evaluated at the productŴ H+1 · · ·Ŵ 1. If we additionally assume that any critical point of 0 is a global minimum, Corollary 5 shows that the empirical risk only has global minima and saddles, and provides a simple condition to distinguish between them. To the best of our knowledge, this is the most general on deep linear networks and it subsumes several previous , e.g., (; ; ; b). This is in Section 4.Notation. For an integer a ≥ 1, [a] denotes the set of integers from 1 to a (inclusive). For a vector v, we use [v] i to denote its i-th component, while [v] [i] denotes a vector comprised of the first i components of v. Let 1 (·) (0 (·) ) be the all ones (zeros) column vector or matrix with size (·). We study below whether nonlinear neural networks provably have spurious local minima. We show in §2 and §3 that even for extremely simple nonlinear networks, one encounters spurious local minima. We first consider ReLU and ReLU-like networks. Here, we prove that as long as linear models cannot perfectly fit the data, there exists a local minimum strictly inferior to the global one. Using nonnegative homogeneity, we can scale the parameters to get infinitely many local minima. Consider a training dataset that consists of m data points. The inputs and the outputs are of dimension d x and d y, respectively. We aggregate these items, and write X ∈ R dx×m as the data matrix and Y ∈ R dy×m as the label matrix. Consider the 1-hidden-layer neural network DISPLAYFORM0. We analyze the empirical risk with squared loss DISPLAYFORM1 F. Next, define a class of piecewise linear nonnegative homogeneous functions h s+,s− (x) = max{s + x, 0} + min{s − x, 0},where s + > 0, s − ≥ 0 and s + = s −. Note that ReLU and Leaky-ReLU are members of this class. We use the shorthandX:= X (C1. DISPLAYFORM0 3) The activation function h ish s+,s−.(C1.4) The hidden layer has at least width 2: DISPLAYFORM1 Then, there is a spurious local minimum whose risk is the same as linear least squares model. Moreover, due to nonnegative homogeneity ofh s+,s−, there are infinitely many such local minima. Noticing that most real world datasets cannot be perfectly fit with linear models, Theorem 1 shows that when we use the activationh s+,s−, the empirical risk has bad local minima for almost all datasets that one may encounter in practice. Although it is not very surprising that neural networks have spurious local minima, proving this rigorously is non-trivial. We provide a constructive and deterministic proof for this problem that holds for general datasets, which is in contrast to experimental of. We emphasize that Theorem 1 also holds even for "slightest" nonlinearities, e.g., when s + = 1 + and s − = 1 where > 0 is small. This suggests that the "local min is global" property is limited to the simplified setting of linear neural networks. Existing on squared error loss either provide one counterexample , or assume realizability and Gaussian input (; BID8 . Realizability is an assumption that the output is generated by a network with unknown parameters. In real datasets, neither input is Gaussian nor output is generated by neural networks; in contrast, our holds for most realistic situations, and hence delivers useful insight. There are several proving sufficient conditions for global optimality of nonlinear neural networks (; ;). But they rely on assumptions that the network width scales with the number of data points. For instance, applying Theorem 3.4 of to our network proves that ifX has linearly independent columns and other assumptions hold, then any critical point with W 2 = 0 is a global minimum. However, linearly independent columns already imply row(X) = R m, so even linear models RX can fit any Y; i.e., there is less merit in using a complex model to fit Y. Theorem 1 does not make any structural assumption other than d 1 ≥ 2, and addresses the case where it is impossible to fit Y with linear models, which is much more realistic. It is worth comparing our with Laurent & Brecht (2018a), who use hinge loss based classification and assume linear separability to prove "no spurious local minima" for Leaky-ReLU networks. Their does not contradict our theorem because the losses are different and we do not assume linear separability. One might wonder if our theorem holds even with d 1 ≥ m. showed that onehidden-layer neural networks with d 1 ≥ m doesn't have spurious valleys, hence there is no strict spurious local minima; however, due to nonnegative homogeneity ofh s+,s− we only have non-strict local minima. Based on BID2, one might claim that with wide enough hidden layer and random W 1 and b 1, one can fit any Y; however, this is not the case, by our assumption that linear models RX cannot fit Y. Note that for any d 1, there is a non-trivial region (measure > 0) in the parameter space where entry-wise). In this region, the output of neural networkŶ is still a linear combination of rows ofX, soŶ cannot fit Y; in fact, it can only do as well as linear models. We will see in the Step 1 of Section 2.2 that the bad local minimum that we construct "kills" d 1 − 1 neurons; however, killing many neurons is not a necessity, and it is just to simply the exposition. In fact, any local minimum in the region W 1 X + b 1 1 T m > 0 is a spurious local minimum. DISPLAYFORM2 The proof of the theorem is split into two steps. First, we prove that there exist local minima (Ŵ j,b j) 2 j=1 whose risk value is the same as the linear least squares solution, and that there are infinitely many such minima. Second, we will construct a tuple of parameters (W j,b j) 2 j=1 that has strictly smaller empirical risk than (Ŵ j,b j) 2 j=1.Step 1: A local minimum as good as the linear solution. The main idea here is to exploit the weights from the linear least squares solution, and to tune the parameters so that all inputs to hidden nodes become positive. Doing so makes the hidden nodes "locally linear," so that the constructed (Ŵ j,b j) 2 j=1 that produce linear least squares estimates at the output become locally optimal. Recall thatX = X T 1 m T ∈ R (dx+1)×m, and define a linear least squares loss 0 (R):= DISPLAYFORM0 T be the output of the linear least squares model, and similarlyȲ =WX.Let η:= min {−1, 2 min iȳi}, a negative constant makingȳ i − η > 0 for all i. Define parameterŝ DISPLAYFORM1 where α > 0 is any arbitrary fixed positive constant, [W] [dx] gives the first d x components ofW, and DISPLAYFORM2 (component-wise), given our choice of η. Thus, all hidden node inputs are positive. Moreover, DISPLAYFORM3 So far, we checked that (Ŵ j,b j) 2 j=1 has the same empirical risk as a linear least squares solution. It now remains to show that this point is indeed a local minimum of. To that end, we consider the perturbed parameters (Ŵ j + ∆ j,b j + δ j) 2 j=1, and check their risk is always larger. A useful point is that sinceW is a minimum of 0 (R) = DISPLAYFORM4 DISPLAYFORM5 they are aggregated perturbation terms. We used to obtain the last equality of. Thus, DISPLAYFORM6 is indeed a local minimum of. Since this is true for arbitrary α > 0, there are infinitely many such local minima. We can also construct similar local minima by permuting hidden nodes, etc. Step 2: A point strictly better than the local minimum. The proof of this step is more involved. In the previous step, we "pushed" all the input to the hidden nodes to positive side, and took advantage of "local linearity" of the hidden nodes near DISPLAYFORM7 j=1 is a spurious local minimum), we make the sign of inputs to the hidden nodes different depending on data. To this end, we sort the indices of data points in increasing order ofȳ i; i.e.,ȳ 1 ≤ȳ 2 ≤ · · · ≤ȳ m. Define the set J: DISPLAYFORM8 The remaining construction is divided into two cases: J = ∅ and J = ∅, whose main ideas are essentially the same. We present the proof for J = ∅, and defer the other case to Appendix A2 as it is rarer, and its proof, while instructive for its perturbation argument, is technically too involved. Case 1: J = ∅. Pick any j 0 ∈ J. We can observe that i≤j0 DISPLAYFORM9, so thatȳ i − β < 0 for all i ≤ j 0 andȳ i − β > 0 for all i > j 0. Then, let γ be a constant satisfying 0 < |γ| ≤ȳ j 0 +1−ȳj 0, whose value will be specified later. Since |γ| is small enough, sign(ȳ i − β) = sign(ȳ i − β + γ) = sign(ȳ i − β − γ). Now select parameters DISPLAYFORM0 Similarly, for i > j 0,ȳ i − β + γ > 0 and −ȳ i + β + γ < 0 inŷ i =ȳ i + s+−s− s++s− γ. Here, we push the outputsŷ i of the network by s+−s− s++s− γ fromȳ i, and the direction of the "push" varies depending on whether i ≤ j 0 or i > j 0.The empirical risk for this choice of parameters is DISPLAYFORM1, and choose small |γ| so that DISPLAYFORM2 is a spurious local minimum. The proof of Theorem 1 crucially exploits the piecewise linearity of the activation functions. Thus, one may wonder whether the spurious local minima seen there are an artifact of the specific nonlinearity. We show below that this is not the case. We provide a counterexample nonlinear network and a dataset for which a wide range of nonlinear activations in a local minimum that is strictly inferior to the global minimum with exactly zero empirical risk. Examples of such activation functions include popular activation functions such as sigmoid, tanh, arctan, ELU, SELU, and ReLU.We consider again the squared error empirical risk of a one-hidden-layer nonlinear neural network: DISPLAYFORM0 F, where we fix d x = d 1 = 2 and d y = 1. Also, let h (k) (x) be the k-th derivative of h: R → R, whenever it exists at x. For short, let h and h denote the first and second derivatives. For this network and dataset the following hold: DISPLAYFORM1 at which equals 0. 2. If there exist real numbers v 1, v 2, u 1, u 2 ∈ R such that the following conditions hold: DISPLAYFORM2 such that the output of the network is the same as the linear least squares model, the risk DISPLAYFORM3 j=1 is a local minimum of.Theorem 2 shows that for this architecture and dataset, activations that satisfy (C2.1)-(C2.7) introduce at least one spurious local minimum. Notice that the empirical risk is zero at the global minimum. This means that the data X and Y can actually be "generated" by the network, which satisfies the realizability assumption that others use (; BID8). Notice that our counterexample is "easy to fit," and yet, there exists a local minimum that is not global. This leads us to conjecture that with harder datasets, the problems with spurious local minima could be worse. The proof of Theorem 2 can be found in Appendix A3.Discussion. Note that the conditions (C2.1)-(C2.7) only require existence of certain real numbers rather than some global properties of activation h, hence are not as restrictive as they look. Conditions (C2.1)-(C2.2) come from a choice of tuple (W j,b j) 2 j=1 that perfectly fits the data. Condition (C2.3) is necessary for constructing (Ŵ j,b j) 2 j=1 with the same output as the linear least squares model, and Conditions (C2.4)-(C2.7) are needed for showing local minimality of (Ŵ j,b j) 2 j=1 via Taylor expansions. The class of functions that satisfy conditions (C2.1)-(C2.7) is quite large, and includes the nonlinear activation functions used in practice. The next corollary highlights this observation (for a proof with explicit choices of the involved real numbers, please see Appendix A5). Corollary 3. For the counterexample in Theorem 2, the set of activation functions satisfying conditions (C2.1)-(C2.7) include sigmoid, tanh, arctan, quadratic, ELU, and SELU.Admittedly, Theorem 2 and Corollary 3 give one counterexample instead of stating a claim about generic datasets. Nevertheless, this example shows that for many practical nonlinear activations, the desirable "local minimum is global" property cannot hold even for realizable datasets, suggesting that the situation could be worse for non-realizable ones. Remark: "ReLU-like" activation functions. Recall the piecewise linear nonnegative homogeneous activation functionh s+,s−. They do not satisfy condition (C2.7), so Theorem 2 cannot be directly applied. Also, if s − = 0 (i.e., ReLU), conditions (C2.1)-(C2.2) are also violated. However, the statements of Theorem 2 hold even forh s+,s−, which is shown in Appendix A6. Recalling again s + = 1 + and s − = 1, this means that even with the "slightest" nonlinearity in activation function, the network has a global minimum with risk zero while there exists a bad local minimum that performs just as linear least squares models. In other words, "local minima are global" property is rather brittle and can only hold for linear neural networks. Another thing to note is that in Appendix A6, the bias parameters are all zero, for both (W j,b j) In this section we present our on deep linear neural networks. Assuming that the hidden layers are at least as wide as either the input or output, we show that critical points of the loss with a multilinear parameterization inherit the type of critical points of the loss with a linear parameterization. As a corollary, we show that for differentiable losses whose critical points are globally optimal, deep linear networks have only global minima or saddle points. Furthermore, we provide an efficiently checkable condition for global minimality. Suppose the network has H hidden layers having widths d 1,..., d H. To ease notation, we set DISPLAYFORM0 The weights between adjacent layers are kept in matrices W j ∈ R dj ×dj−1 DISPLAYFORM1, and the outputŶ of the network is given by the product of weight matrices with the data matrix: DISPLAYFORM2 j=1 be the tuple of all weight matrices, and W i:j denote the product W i W i−1 · · · W j+1 W j for i ≥ j, and the identity for i = j − 1. We consider the empirical risk ((W j) H+1 j=1 ), which, for linear networks assumes the form DISPLAYFORM3 where 0 is a suitable differentiable loss. For example, when 0 (R) = DISPLAYFORM4 Remark: bias terms. We omit the bias terms b 1,..., b H+1 here. This choice is for simplicity; models with bias can be handled by the usual trick of augmenting data and weight matrices. We are now ready to state our first main theorem, whose proof is deferred to Appendix A7. Theorem 4. Suppose that for all j, d j ≥ min{d x, d y}, and that the loss is given by, where 0 is differentiable on R dy×dx. For any critical point (Ŵ j) H+1 j=1 of the loss, the following claims hold: DISPLAYFORM0 j=1 is a saddle of. DISPLAYFORM1 j=1 is a local min (max) of ifŴ H+1:1 is a local min (max) of 0; moreover, DISPLAYFORM2 j=1 is a global min (max) of if and only ifŴ H+1:1 is a global min (max) of 0.3. If there exists j * ∈ [H + 1] such thatŴ H+1:j * +1 has full row rank andŴ j * −1:1 has full column rank, then ∇ 0 (Ŵ H+1:1) = 0, so 2(a) and 2(b) hold. Also, DISPLAYFORM3 j=1 is a local min (max) of.Let us paraphrase Theorem 4 in words. In particular, it states that if the hidden layers are "wide enough" so that the product W H+1:1 can attain full rank and if the loss assumes the form for a differentiable loss 0, then the type (optimal or saddle point) of a critical point (Ŵ j) H+1 j=1 of is governed by the behavior of 0 at the productŴ H+1:1.Note that for any critical point (Ŵ j) H+1 j=1 of the loss, either ∇ 0 (Ŵ H+1:1) = 0 or ∇ 0 (Ŵ H+1:1) = 0. Parts 1 and 2 handle these two cases. Also observe that the condition in Part 3 implies ∇ 0 = 0, so Part 3 is a refinement of Part 2. A notable fact is that a sufficient condition for Part 3 isŴ H+1:1 having full rank. For example, if d x ≥ d y, full-rankŴ H+1:1 implies rank(Ŵ H+1:2) = d y, whereby the condition in Part 3 holds with j * = 1.IfŴ H+1:1 is not critical for 0, then (Ŵ j) H+1 j=1 must be a saddle point of. IfŴ H+1:1 is a local min/max of 0, (Ŵ j) H+1 j=1 is also a local min/max of. Notice, however, that Part 2(a) does not address the case of saddle points; whenŴ H+1:1 is a saddle point of 0, the tuple (Ŵ j) H+1 j=1 can behave arbitrarily. However, with the condition in Part 3, statements 2(a) and 3(a) hold at the same time, so thatŴ H+1:1 is a local min/max of 0 if and only if (Ŵ j) H+1 j=1 is a local min/max of. Observe that the same "if and only if" statement holds for saddle points due to their definition; in summary, the types (min/max/saddle) of the critical points (Ŵ j) H+1 j=1 andŴ H+1:1 match exactly. Although Theorem 4 itself is of interest, the following corollary highlights its key implication for deep linear networks. Corollary 5. In addition to the assumptions in Theorem 4, assume that any critical point of 0 is a global min (max). For any critical point (Ŵ j) Corollary 5 shows that for any differentiable loss function 0 whose critical points are global minima, the loss has only global minima and saddle points, therefore satisfying the "local minima are global" property. In other words, for such an 0, the multilinear re-parametrization introduced by deep linear networks does not introduce any spurious local minima/maxima; it only introduces saddle points. Importantly, Corollary 5 also provides a checkable condition that distinguishes global minima from saddle points. Since is nonconvex, it is remarkable that such a simple necessary and sufficient condition for global optimality is available. DISPLAYFORM4 Our generalizes previous works on linear networks such as Kawaguchi FORMULA2;; , because it provides conditions for global optimality for a broader range of loss functions without assumptions on datasets. Laurent & Brecht (2018b) proved that if DISPLAYFORM5 j=1 is a local min of, thenŴ H+1:1 is a critical point of 0. First, observe that this is implied by Theorem 4.1. So our , which was proved in parallel and independently, is strictly more general. With additional assumption that critical points of 0 are global minima, Laurent & Brecht (2018b) showed that "local min is global" property holds for linear neural networks; our Corollay 5 gives a simple and efficient test condition as well as proving there are only global minima and saddles, which is clearly stronger. We investigated the loss surface of deep linear and nonlinear neural networks. We proved two theorems showing existence of spurious local minima on nonlinear networks, which apply to almost all datasets (Theorem 1) and a wide class of activations (Theorem 2). We concluded by Theorem 4, showing a general studying the behavior of critical points in multilinearly parametrized functions, which unifies other existing on linear neural networks. Given that spurious local minima are common in neural networks, a valuable future research direction will be investigating how far local minima are from global minima in general, and how the size of the network affects this gap. Another thing to note is that even though we showed the existence of spurious local minima in the whole parameter space, things can be different in restricted sets of parameter space (e.g., by adding regularizers). Understanding the loss surface in such sets would be valuable. Additionally, one can try to show algorithmic/trajectory of (stochastic) gradient descent. We hope that our paper will be a stepping stone to such future research. A2 PROOF OF THEOREM 1, STEP 2, CASE 2 Case 2. J = ∅. We start with a lemma discussing what J = ∅ implies. Lemma A.1. If J = ∅, the following statements hold:1. There are someȳ j's that are duplicate; i.e. for some i = j,ȳ i =ȳ j.2. Ifȳ j is non-duplicate, meaning thatȳ j−1 <ȳ j <ȳ j+1,ȳ j = y j holds.3. Ifȳ j is duplicate, i:ȳi=ȳj (ȳ i − y i) = 0 holds.4. There exists at least one duplicateȳ j such that, for thatȳ j, there exist at least two different i's that satisfyȳ i =ȳ j andȳ i = y i.Proof We prove this by showing if any of these statements are not true, then we have J = ∅ or a contradiction.1. If all theȳ j's are distinct and J = ∅, by definition of J,ȳ j = y j for all j. This violates our assumption that linear models cannot perfectly fit Y.2. If we haveȳ j = y j for a non-duplicateȳ j, at least one of the following statements must hold: i≤j−1 (ȳ i − y i) = 0 or i≤j (ȳ i − y i) = 0, meaning that j − 1 ∈ J or j ∈ J.3. Supposeȳ j is duplicate and i:ȳi=ȳj (ȳ i − y i) = 0. Let k = min{i |ȳ i =ȳ j} and l = max{i |ȳ i =ȳ j}. Then at least one of the following statements must hold: DISPLAYFORM0 4. Since i:ȳi=ȳj (ȳ i − y i) = 0 holds for any duplicateȳ j, ifȳ i = y i holds for one i then there must be at least two of them that satisfiesȳ i = y i. If this doesn't hold for all duplicatē y i, with Part 2 this means thatȳ j = y j holds for all j. This violates our assumption that linear models cannot perfectly fit Y.From Lemma A.1.4, we saw that there is a duplicate value ofȳ j such that some of the data points i satisfyȳ i =ȳ j andȳ i = y i. The proof strategy in this case is essentially the same, but the difference is that we choose one of such duplicateȳ j, and then choose a vector v ∈ R dx to "perturb" the linear least squares solution [W] [dx] in order to break the tie between i's that satisfiesȳ i =ȳ j andȳ i = y i.We start by defining the minimum among such duplicate valuesȳ * ofȳ j's, and a set of indices j that satisfiesȳ j =ȳ *.ȳ * = min{ȳ j | ∃i = j such thatȳ i =ȳ j andȳ i = y i}, DISPLAYFORM1 Then, we define a subset of J *: DISPLAYFORM2 By Lemma A.1.4, cardinality of J * = is at least two. Then, we define a special index in J * =: DISPLAYFORM3 Index j 1 is the index of the "longest" x j among elements in J * =. Using the definition of j 1, we can partition J * into two sets: DISPLAYFORM4 2 2 }. For the indices in J *, we can always switch the indices without loss of generality. So we can assume that j ≤ j 1 = max J * ≥ for all j ∈ J * ≥ and j > j 1 for all j ∈ J * <.We now define a vector that will be used as the "perturbation" to [W] [dx]. Define a vector v ∈ R dx, which is a scaled version of x j1: DISPLAYFORM5 where the constants g and M are defined to be DISPLAYFORM6 The constant M is the largest x i 2 among all the indices, and g is one fourth times the minimum gap between all distinct values ofȳ i. [dx] by a vector −αv T. where α ∈ will be specified later. Observe that DISPLAYFORM0 Recall that j ≤ j 1 = max J * ≥ for all j ∈ J * ≥ and j > j 1 for all j ∈ J * <. We are now ready to present the following lemma: Lemma A.2. Define DISPLAYFORM1 Proof First observe that, for any x i, |αv DISPLAYFORM2 By definition of g, we have 2g <ȳ j −ȳ i for anyȳ i <ȳ j. Using this, we can see that DISPLAYFORM3 In words, ifȳ i andȳ j are distinct and there is an orderȳ i <ȳ j, perturbation of [W] [dx] by −αv T does not change the order. Also, since v is only a scaled version of x j1, from the definitions of J * ≥ and J * DISPLAYFORM4 It is left to prove the statement of the lemma using case analysis, using the inequalities (A.1), (A.2), and (A.3). For all i's such thatȳ i <ȳ * =ȳ j1, DISPLAYFORM5 Similarly, for all i such thatȳ i >ȳ * =ȳ j2, DISPLAYFORM6 For j ∈ J * ≥ (j ≤ j 1), we knowȳ j =ȳ *, sō DISPLAYFORM7 Also, for j ∈ J * < (j > j 1), DISPLAYFORM8 This finishes the case analysis and proves the first statements of the lemma. One last thing to prove is that i>j1 DISPLAYFORM9 Recall from Lemma A.1.2 that for non-duplicateȳ j, we haveȳ j = y j. Also by Lemma A.1.3 ifȳ j is duplicate, DISPLAYFORM10 Recall the definition of J * = = {j ∈ J * |ȳ j = y j}. For j ∈ J * \J * =,ȳ j = y j. So, DISPLAYFORM11 Recall the definition of j 1 = argmax j∈J * = x j 2. For any other j ∈ J * = \{j 1}, DISPLAYFORM12, where the first ≥ sign is due to definition of j 1, and the second is from Cauchy-Schwarz inequality. Since x j1 and x j are distinct by assumption, they must differ in either length or direction, or both. So, we can check that at least one of "≥" must be strict inequality, so x j1 2 2 > x j, x j1 for all j ∈ J * = \{j 1}. Thus, DISPLAYFORM13 Also, by Lemma A.1.3, DISPLAYFORM14 Wrapping up all the equalities, we can conclude that DISPLAYFORM15 finishing the proof of the last statement. It is time to present the parameters (W j,b j) 2 j=1, whose empirical risk is strictly smaller than the local minimum (Ŵ j,b j) 2 j=1 with a sufficiently small choice of α ∈. Now, let γ be a constant such that DISPLAYFORM16 Its absolute value is proportional to α ∈, which is a undetermined number that will be specified at the end of the proof. Since |γ| is small enough, we can check that sign(ȳ i − αv DISPLAYFORM17 Then, assign parameter values DISPLAYFORM18 With these parameter values,W DISPLAYFORM19 As we saw in Lemma A.2, for i ≤ j 1,ȳ i − αv DISPLAYFORM20 Similarly, for i > j 1,ȳ i − αv DISPLAYFORM21 Now, the squared error loss of this point is DISPLAYFORM22 Recall that DISPLAYFORM23 As seen in the definition of γ (A.4), the magnitude of γ is proportional to α. Substituting (A.4), we can express the loss as DISPLAYFORM24 Recall that v T (x j1 − x j2) > 0 from (A.3). Then, for sufficiently small α ∈, DISPLAYFORM25 DISPLAYFORM26 With these values, we can check thatŶ = , hence perfectly fitting Y, thus the loss DISPLAYFORM27 Given conditions (C2.3)-(C2.7) on v 1, v 2, u 1, u 2 ∈ R, we prove below that there exists a local minimum (Ŵ j,b j) 2 j=1 for which the output of the network is the same as linear least squares model, and its empirical risk is ((Ŵ j,b j) 2 j=1 ) =. Now assign parameter valuesŴ DISPLAYFORM0 With these values we can check thatŶ =. It remains to show that this is indeed a local minimum of. To show this, we apply perturbations to the parameters to see if the risk after perturbation is greater than or equal to ((Ŵ j,b j) 2 j=1 ). Let the perturbed parameters bě DISPLAYFORM1 where δ 11, δ 12, δ 21, δ 22, β 1, β 2, 1, 2, and γ are small real numbers. The next lemma rearranges the terms in ((W j,b j) 2 j=1 ) into a form that helps us prove local minimality of (Ŵ j,b j) 2 j=1. Appendix A4 gives the proof of Lemma A.3, which includes as a byproduct some equalities on polynomials that may be of wider interest. Lemma A.3. Assume there exist real numbers v 1, v 2, u 1, u 2 such that conditions (C2.3)-(C2.5) hold. Then, for perturbed parameters DISPLAYFORM2 where DISPLAYFORM3 + o, for i = 1, 2, and DISPLAYFORM4 + o, and o contains terms that diminish to zero as perturbations vanish. To make the the sum of the last three terms of (A.6) nonnegative, we need to satisfy α 1 ≥ 0 and α 2 3 − 4α 1 α 2 ≤ 0; these inequalities are satisfied for small enough perturbations because of conditions (C2.6)-(C2.7). Thus, we conclude that DISPLAYFORM5 j=1 is a local minimum. The goal of this lemma is to prove that DISPLAYFORM0 where o contains terms that diminish to zero as perturbations decrease. Using the perturbed parameters, DISPLAYFORM1 so the empirical risk can be expressed as DISPLAYFORM2 (A.8) So, the empirical risk (A.8) consists of three terms, one for each training example. By expanding the activation function h using Taylor series expansion and doing algebraic manipulations, we will derive the equation (A.7) from (A.8).Using the Taylor series expansion, we can express h(v 1 + δ 11 + β 1) as DISPLAYFORM3 Using a similar expansion for h(v 2 + δ 21 + β 2), the first term of (A.8) can be written as DISPLAYFORM4 where we used u 1 h(v 1)+u 2 h(v 2) = 1 3. To simplify notation, let us introduce the following function: DISPLAYFORM5 With this new notation t(δ 1, δ 2), after doing similar expansions to the other terms of (A.8), we get DISPLAYFORM6 Before we show the lower bounds, we first present the following lemmas that will prove useful shortly. These are simple yet interesting lemmas that might be of independent interest. Lemma A.4. For n ≥ 2, DISPLAYFORM7 where p n is a polynomial in a and b. All terms in p n have degree exactly n − 2. When n = 2, DISPLAYFORM8 Proof The exact formula for p n (a, b) is as the following: DISPLAYFORM9 Using this, we can check the lemma is correct just by expanding both sides of the equation. The rest of the proof is straightforward but involves some complicated algebra. So, we omit the details for simplicity. Lemma A.5. For n 1, n 2 ≥ 1, DISPLAYFORM10 where q n1,n2 and r n1,n2 are polynomials in a, b, c and d. All terms in q n1,n2 and r n1,n2 have degree exactly DISPLAYFORM11 Proof The exact formulas for q n1,n2 (a, b, d), q n2,n1 (c, d, b), and r n1,n2 (a, b, c, d) are as the following: DISPLAYFORM12 Similarly, we can check the lemma is correct just by expanding both sides of the equation. The remaining part of the proof is straightforward, so we will omit the details. Using Lemmas A.4 and A.5, we will expand and simplify the "cross terms" part and "squared terms" part of (A.9). For the "cross terms" in (A.9), let us split t(δ 1, δ 2) into two functions t 1 and t 2: DISPLAYFORM13 It is easy to check that DISPLAYFORM14 Also, using Lemma A.4, we can see that DISPLAYFORM15 Consider the summation DISPLAYFORM16 We assumed that there exists a constant c > 0 such that |h (n) (v 1)| ≤ c n n!. From this, for small enough perturbations δ 11, δ 12, and β 1, we can see that the summation converges, and the summands converge to zero as n increases. Because all the terms in p n (n ≥ 3) are of degree at least one, we can thus write DISPLAYFORM17 So, for small enough δ 11, δ 12, and β 1, the term DISPLAYFORM18 dominates the summation. Similarly, as long as δ 21, δ 22, and β 2 are small enough, the summation DISPLAYFORM19. In , for small enough perturbations, DISPLAYFORM20 Now, it is time to take care of the "squared terms." We will express the terms as This time, we split t(δ 1, δ 2) in another way, this time into three parts: DISPLAYFORM21 so that t(δ 1, δ 2) = t 3 + t 4 (δ 1) + t 5 (δ 2). DISPLAYFORM22 as seen in (A.10). Next, we have (A.14) when perturbations are small enough. We again used Lemma A.4 in the second equality sign, and the facts that p n1+n2 (·) = o whenever n 1 + n 2 > 2 and that p 2 (·) = 1 2. In a similar way, DISPLAYFORM23 DISPLAYFORM24 Lastly, A.16) where the second equality sign used Lemma A.5 and the third equality sign used the facts that q n1,n2 (·) = o and r n1,n2 (·) = o whenever n 1 + n 2 > 2, and that q 1,1 (·) = 0 and r 1,1 (·) = 1 2. If we substitute (A.13), (A.14), (A.15), and (A.16) into (A.12), DISPLAYFORM25 DISPLAYFORM26 We are almost done. If we substitute (A.10), (A.11), and (A.17) into (A.9), we can get DISPLAYFORM27 which is the equation (A.7) that we were originally aiming to show. For the proof of this corollary, we present the values of real numbers that satisfy assumptions (C2.1)-(C2.7), for each activation function listed in the corollary: sigmoid, tanh, arctan, exponential linear units , scaled exponential linear units .To remind the readers what the assumptions were, we list the assumptions again. For (C2.1)-(C2.2), there exist real numbers v 1, v 2, v 3, v 4 ∈ R such that DISPLAYFORM0 For (C2.3)-(C2.7), there exist real numbers v 1, v 2, u 1, u 2 ∈ R such that the following assumptions hold: DISPLAYFORM1 ).For each function, we now present the appropriate real numbers that satisfy the assumptions. When h is sigmoid, DISPLAYFORM0 Assumptions (C2.1)-(C2.2) are satisfied by DISPLAYFORM1 and assumptions (C2.3)-(C2.7) are satisfied by DISPLAYFORM2 Among them, (C2.4)-(C2.5) follow because sigmoid function is an real analytic function Krantz & Parks (2002 When h is quadratic, assumptions (C2.1)-(C2.2) are satisfied by DISPLAYFORM0 and assumptions (C2.3)-(C2.7) are satisfied by DISPLAYFORM1 In the case of s − > 0, assumptions (C2.1)-(C2.2) are satisfied by DISPLAYFORM2 The rest of the proof can be done in exactly the same way as the proof of Theorem 2.1, provided in Appendix A3.For s − = 0, which corresponds to the case of ReLU, define parameters DISPLAYFORM3 We can check thath DISPLAYFORM4 Assumptions (C2.3)-(C2.6) are satisfied by DISPLAYFORM0 Assign parameter valueŝ DISPLAYFORM1 It is easy to compute that the output of the neural network isŶ =. Now, it remains to show that this is indeed a local minimum of. To show this, we apply perturbations to the parameters to see if the risk after perturbation is greater than or equal to ((Ŵ j,b j) 2 j=1 ). Let the perturbed parameters bě DISPLAYFORM2 where δ 11, δ 12, δ 21, δ 22, β 1, β 2, 1, 2, and γ are small enough real numbers. Using the perturbed parameters, DISPLAYFORM3 so the empirical risk can be expressed as DISPLAYFORM4 Published as a conference paper at ICLR 2019To simplify notation, let us introduce the following function: DISPLAYFORM5 It is easy to check that DISPLAYFORM6 Before we start, note the following partial derivatives, which can be computed using straightforward matrix calculus: DISPLAYFORM0 For Part 1, we must show that if ∇ 0 (Ŵ H+1:1) = 0 then (Ŵ j) j=1 is a saddle point of. Thus, we show that (Ŵ j) H+1 j=1 is neither a local minimum nor a local maximum. More precisely, for each j, let B (W j) be an -Frobenius-norm-ball centered at W j, and H+1 j=1 B (W j) their Cartesian product. We wish to show that for every > 0, there exist tuples (P j) DISPLAYFORM0 To prove (A.18), we exploit ((Ŵ j) H+1 j=1 ) = 0 (Ŵ H+1:1), and the assumption ∇ 0 (Ŵ H+1:1) = 0. The key idea is to perturb the tuple (Ŵ j) H+1 j=1 so that the directional derivative of 0 along P H+1:1 − W H+1:1 is positive. Since 0 is differentiable, if P H+1:1 −Ŵ H+1:1 is small, then DISPLAYFORM1 Similarly, we can show ((Q j) H+1 j=1 ) < ((Ŵ j) H+1 j=1 ). The key challenge lies in constructing these perturbations; we outline our approach below; this construction may be of independent interest too. For this section, we assume that d x ≥ d y for simplicity; the case d y ≥ d x is treated in Appendix A7.2.Since ∇ 0 (Ŵ H+1:1) = 0, col(∇ 0 (Ŵ H+1:1)) ⊥ must be a strict subspace of R dy. Consider ∂ /∂W 1 at a critical point to see that (Ŵ H+1:2) T ∇ 0 (Ŵ H+1:1) = 0, so col(Ŵ H+1:2) ⊆ col(∇ 0 (Ŵ H+1:1)) ⊥ R dy. This strict inclusion implies rank(Ŵ H+1:2) < d y ≤ d 1, so that null(Ŵ H+1:2) is not a trivial subspace. Moreover, null(Ŵ H+1:2) ⊇ null(Ŵ H:2) ⊇ · · · ⊇ null(Ŵ 2). We can split the proof into two cases: null(Ŵ H+1:2) = null(Ŵ H:2) and null(Ŵ H+1:2) = null(Ŵ H:2). Recall that W 1 ∈ R d1×dx. Given R ∈ R dy×dx, we can fill the first d y rows of W 1 with R and let any other entries be zero. For all the other matrices W 2,..., W H+1, we put ones to the diagonal entries while putting zeros to all the other entries. We can check that, by this construction, R = W H+1:1 for this given R. DISPLAYFORM2 dy×dx, we can fill the first d x columns of W H+1 with R and let any other entries be zero. For all the other matrices W 1,..., W H, we put ones to the diagonal entries while putting zeros to all the other entries. By this construction, R = W H+1:1 for given R.Once this fact is given, by ((W j) ). To show thatŴ H+1:1 is a local min of 0, we have to show there exists a neighborhood ofŴ H+1:1 such that, any point R in that neighborhood satisfies 0 (R) ≥ 0 (Ŵ H+1:1). To prove this, we state the following lemma: Lemma A.6. Suppose A:=Ŵ H+1:j * +1 has full row rank and B:=Ŵ j * −1:1 has full column rank. Then, any R satisfying R −Ŵ H+1:1 F ≤ σ min (A)σ min (B) can be decomposed into R = V H+1:1, where DISPLAYFORM3 and V j =Ŵ j for j = j *. Also, V j −Ŵ j F ≤ for all j. Proof Since A:=Ŵ H+1:j * +1 has full row rank and B:=Ŵ j * −1:1 has full column rank, σ min (A) > 0, σ min (B) > 0, and AA T and B T B are invertible. Consider any R satisfying R −Ŵ H+1:1 F ≤ σ min (A)σ min (B). Given the definitions of V j's in the statement of the lemma, we can check the identity that R = V H+1:1 by Therefore, DISPLAYFORM4 · σ min (A)σ min (B) =.The lemma shows that for any R = V H+1:1 satisfying R −Ŵ H+1:1 F ≤ σ min (A)σ min (B), we have 0 (R) = 0 (V H+1:1) = ((V j) H+1 j=1 ) ≥ ((Ŵ j) H+1 j=1 ) = 0 (Ŵ H+1:1). We can prove the local maximum part by a similar argument."

We introduce a simple procedure to repurpose pre-trained transformer-based language models to perform abstractive summarization well.

Represent the TLDR for finding the corresponding scientific article

We provide insightful understanding of sequence-labeling NER and propose to use two types of cross structures, both of which bring theoretical and empirical improvements.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: This paper addresses the problem of incremental domain adaptation (IDA). We assume each domain comes sequentially, and that we could only access data in the current domain. The goal of IDA is to build a unified model performing well on all the encountered domains. We propose to augment a recurrent neural network (RNN) with a directly parameterized memory bank, which is retrieved by an attention mechanism at each step of RNN transition. The memory bank provides a natural way of IDA: when adapting our model to a new domain, we progressively add new slots to the memory bank, which increases the model capacity. We learn the new memory slots and fine-tune existing parameters by back-propagation. Experiments show that our approach significantly outperforms naive fine-tuning and previous work on IDA, including elastic weight consolidation and the progressive neural network. Compared with expanding hidden states, our approach is more robust for old domains, shown by both empirical and theoretical . Domain adaptation aims to transfer knowledge from a source domain to a target domain in a machine learning system. This is important for neural networks, which are data-hungry and prone to overfitting. In this paper, we focus on incremental domain adaptation (IDA), where we assume different domains come one after another. We only have access to the data in the current domain, but hope to build a unified model that performs well on all the domains that we have encountered (; ;).Incremental domain adaptation is useful in various scenarios. Suppose a company is doing business with different partners over a long period of time. The company can only access the data of the partner with a current contract. However, the machine learning model is the company's property (if complying with the contract). Therefore, it is desired to preserve as much knowledge as possible in the model and not to rely on the availability of the data. Another important application of IDA is a quick adaptation to new domains. If the environment of a deployed machine learning system changes frequently, traditional methods like jointly training all domains require the learning machine to be re-trained from scratch every time. Fine-tuning a neural network by a few steps of gradient updates does transfer quickly, but it suffers from the catastrophic forgetting problem . Suppose we do not know the domain of a data point when predicting; the (single) finetuned model cannot predict well for samples in previous domains, as it tends to "forget" quickly during fine-tuning. A recent trend of domain adaptation in the deep learning regime is the progressive neural network , which progressively grows the network capacity if a new domain comes. Typically, this is done by enlarging the model with new hidden states and a new predictor (FIG0). To avoid interfering with existing knowledge, the newly added hidden states are not fed back to the previously trained states. During training, they fix all existing parameters, and only train the newly added ones. For inference, they use the new predictor for all domains. This is sometimes undesired as the new predictor is trained with only the last domain. In this paper, we propose a progressive memory bank for incremental domain adaptation. Our model augments a recurrent neural network (RNN) with a memory bank, which is a set of distributed, real-valued vectors capturing domain knowledge. The memory is retrieved by an attention mechanism. When our model is adapted to new domains, we progressively increase the slots in the memory bank. But different from , we fine-tune all the parameters, including RNN and the existing memory slots. Empirically, when the model capacity increases, the RNN does not forget much even if the entire network is fine-tuned. Compared with expanding RNN hidden states, the newly added memory slots do not contaminate existing knowledge in RNN states, as will be shown by a theorem. We evaluate our approach 1 on Natural Language Inference and Dialogue Response Generation. Experiments support our hypothesis that the proposed approach adapts well to target domains without catastrophic forgetting of the source. Our model outperforms the naïve fine-tuning method, the original progressive neural network, as well as other IDA techniques including elastic weight consolidation (EWC) .Detailed related work is provided in Appendix A. Our model is based on an RNN. At each time step, the RNN takes the embedding of the current word as input, and changes its states accordingly. This can be represented by DISPLAYFORM0 where h i and h i−1 are the hidden states at time steps i and i − 1, respectively. x i is the input at the ith step. Typically, long short term memory (LSTM) or Gated Recurrent Units (GRU) are used as RNN transitions. In the rest of this section, we will describe a memory augmented RNN, and how it is used for incremental domain adaptation (IDA). We enhance the RNN with an external memory bank, as shown in FIG0. The memory bank augments the overall model capacity by storing additional parameters in memory slots. At each time step, our model computes an attention probability to retrieve memory content, which is then fed to the computation of RNN transition. Particularly, we adopt a key-value memory bank, inspired by. Each memory slot contains a key vector and a value vector. The former is used to compute the attention weight for memory retrieval, whereas the latter is the value of memory content. For the ith step, the memory mechanism computes an attention probability α i by DISPLAYFORM0 where m (key) j is the key vector of the jth slot of the memory (among N slots in total). Then the model retrieves memory content by a weighted sum of all memory values, where the weight is the attention probability, given by DISPLAYFORM1 is the value vector of the jth memory slot. We call c i the memory content. Then, c i is concatenated with the current word x i, and fed to the RNN at step i to compute RNN state transition. The memory bank in our model captures distributed knowledge; this is different from other work where memory slots correspond to specific entities . The attention mechanism enables us to train both memory content and its retrieval end-to-end, along with other parameters. The memory bank in Subsection 2.1 can be progressively expanded to adapt a model in a source domain to new domains. This is done by adding new memory slots to the bank which are learned exclusively from the target data. Suppose the memory bank is expanded with another M slots in a new domain, in addition to previous N slots. We then have N + M slots in total. The model computes attention probability over the expanded memory and obtains the attention vector in the same way as Equations -, except that the summation is computed from 1 to N + M. To initialize the expanded model, we load all previous parameters, including RNN weights and the learned N slots, but randomly initialize the progressively expanded M slots. During training, we update all parameters by gradient descent. The process is applied whenever a new domain comes, as shown in Algorithm 1 in Appendix A.We would like to discuss the following issues. DISPLAYFORM0 where h We evaluate our approach on natural language inference. This is a classification task to determine the relationship between two sentences, the target labels being entailment, contradiction, and neutral. we train a bi-directional LSTM (BiLSTM), following the original MultiNLI paper . Our BiL-STM achieves an accuracy of 68.37 on the official MultiNLI test set, which is better than 67.51 reported in the original MultiNLI paper using BiLSTM. This shows that our implementation and tuning are fair for the basic BiLSTM, and that our model is ready for the study of IDA. The details of network architecture, training and hyper-parameter tuning are given in Appendix C. We want to compare our approach with a large number of baselines and variants, and thus choose two domains as a testbed: Fic as the source and Gov as the target. We show in Table 1.First, we analyze the performance of RNN and the memoryaugmented RNN (Lines 1-2 vs. Lines 3-4). They have generally similar performance, showing that, in the non-transfer setting, the memory bank does not help the RNN much, and thus is not a typical RNN architecture in previous literature. However, This later confirms that the performance improvement is indeed due to our IDA technique, instead of simply a better neural architecture. We then apply two straightforward methods of domain adaptation: multi-task learning (Line 5) and fine-tuning (Line 6). Multi-task learning jointly optimizes source and target objectives, denoted by "S+T." On the other hand, the fine-tuning approach trains the model on the source first, and then finetunes on the target. In our experiments, these two methods perform similarly on the target domain, which is consistent with (performs significantly worse than multi-task learning, as it suffers from the catastrophic forgetting problem. We notice that, in terms of source performance, the fine-tuning approach (Line 6) is slightly better than trained on the source domain only (Line 3). This is probably because our domains are highly correlated as opposed to , and thus training with more data on target improves the performance on source. However, fine-tuning does achieve the worst performance on source compared with other domain adaptation approaches (among Lines 5-8). Thus, we nevertheless use the terminology "catastrophic forgetting", and our research goal is still to improve IDA performance. The main of our approach are Lines 7 and 8. We see that on both source and target domains, our approach outperforms the fine-tuning method alone where the memory size is not increased (comparing Lines 7 and 6). This verifies our conjecture that, if the model capacity is increased sufficiently, the new domain does not override the learned knowledge much in the neural network. Our proposed approach is also "orthogonal" to the expansion of the vocabulary size, where target-specific words are randomly initialized and learned on the target domain. As seen, this combines well with our memory expansion and yields the best performance on both source and target (Line 8).We now compare an alternative way of increasing model capacity, i.e., expanding hidden states (Lines 9 and 10). For fair comparison, we ensure that the total number of model parameters after memory expansion is equal to the number of model parameters after hidden state expansion. We see that the performance of hidden state expansion is poor especially on the source domain, even if we fine-tune all parameters. This experiment provides empirical evidence to our theorem that expanding memory is more robust than expanding hidden states. We also compare the with previous work on IDA. EWC does not achieve satisfactory . We investigate other published papers using the same method and find inconsistent : EWC works well in some applications but performs poorly on others ; even report near random performance with EWC. We also re-implement the progressive neural network . We use the target predictor to do inference for both source and target domains. Progressive neural network also yields low performance, particularly on source, probably because the predictor is trained with only the target domain. We measure the statistical significance of the with one-tailed Wilcoxon's signed-rank test . Each method is compared with Line 8: ↑ and ⇑ denote "significantly better" with p < 0.05 and p < 0.01 respectively. ↓ and ⇓ similarly denote "significantly worse". The absence of an arrow indicates that the performance difference compared with Line 8 is statistically insignificant with p < 0.05. The test shows that our approach is significantly better than others, both on source and target. Having analyzed our approach, baselines, and variants on two domains in detail, we test the performance of IDA with multiple domains, namely, Fic, Gov, Slate, Tel, and Travel. We assume these domains come one after another, and our goal is to achieve high performance on both new and previous domains. TAB3 shows that our approach of progressively growing memory bank achieves the same performance as fine-tuning on the last domain (both with vocabulary expansion). But for all previous 4 domains, we achieve significantly better performance. Our model is comparable to multi-task learning on all domains. It also outperforms EWC and the progressive neural network in all domains; the are consistent with Table 1. This provides evidence of the effectiveness for IDA with more than two domains. It should also be mentioned that multi-task learning requires data from all domains to be available at the same time. It is not an incremental approach for domain adaptation, and thus cannot be applied to the scenarios introduced in Section 1. We include this setting mainly because we are curious about the performance of non-incremental domain adaptation. In this paper, we propose a progressive memory network for incremental domain adaptation (IDA). We augment an RNN with an attention-based memory bank. During IDA, we add new slots to the memory bank and tune all parameters by back-propagation. Empirically, the progressive memory network does not suffer from the catastrophic forgetting problem as in naïve fine-tuning. Our intuition is that the new memory slots increase the neural network's model capacity, and thus, the new knowledge less overrides the existing network. Compared with expanding hidden states, our progressive memory bank provides a more robust way of increasing model capacity, shown by both a theorem and experiments. We also outperform previous work for IDA, including elastic weight consolidation (EWC) and the original progressive neural network. Bayer, J., Osendorfer, C., Korhammer, D., Chen, N., Urban, S., and van der Smagt, P. On fast dropout and its applicability to recurrent networks. Rusu et al. FORMULA0 propose a progressive neural network that progressively increases the number of hidden states (FIG0). To avoid overriding existing information, they propose to fix the weights of the learned network, and do not feed new states to old ones. This in multiple predictors, requiring that a data sample is labeled with its domain during the test time. Should different domains be highly correlated to each other, the predictor of a previous domain cannot make use of new data to improve performance. If we otherwise use the last predictor to predict samples from all domains, its performance may be low for previous domains, as the predictor is only trained with the last domain. propose an extension of the progressive network. They identify which existing hidden units are relevant for the new task (with their sparse penalty), and finetune only the corresponding subnetwork. However, sparsity is not common for RNNs in NLP applications, as sparse recurrent connections are harmful. A similar phenomenon is that dropout of recurrent connections is harmful . Our work is related to memory-based neural networks. propose an end-to-end memory network that assigns a slot for an entity, and aggregates information by multiple attention-based layers. In their work, they design the architecture for bAbI question answering, and assign a memory slot for each sentence. Such idea can be extended to various scenarios, for example, assigning slots to external knowledge for question answering and assigning slots to dialog history for a conversation system .Another type of memory in the neural network regime is the neural Turing machine (NTM) . Their memory is not directly parameterized, but is read or written by a neural controller. Therefore, such memory serves as temporary scratch paper, but does not store knowledge itself. In NTM, the memory information and operation are fully distributed/neuralized, as they do not correspond to the program on a true (non-neural) Turing machine. Zhang et al. (2018b) combine the above two styles of memory for task-oriented dialog systems, where they have both slot-value memory and read-and-write memory. Different from the above work, our memory bank stores knowledge in a distributed fashion, where each slot does not correspond to a concrete entity. Our memory is directly parameterized, interacting in a different way from RNN weights. Thus, it provides a natural way of incremental domain adaptation. Our proposed IDA process is shown in Algorithm 1. It is noted that the following theorem does not explicitly prove for IDA, but shows that expanding memory is more stable than expanding hidden states. This is particularly important at the beginning steps of IDA, as the progressively growing parameters are randomly initialized and are basically noise. Although our theoretical analysis uses a restricted setting (i.e., vanilla RNN transition and linear activation), it provides the key insight that our approach is appropriate for IDA. tions. That is, DISPLAYFORM0 where h Proof: We first make a few assumptions. Let h i−1 be the hidden state of the last step. We focus on one step of transition and assume that h i−1 is the same when the model capacity is increased. We consider a simplified case where the RNN has vanilla transition with the linear activation function. We measure the effect of model expansion quantitatively by the expected norm of the difference on h i before and after model expansion. Suppose the original hidden state h i is D-dimensional. We assume each memory slot is d-dimensional, and that the additional RNN units when expanding the hidden state are also d-dimensional. We further assume every variable in the expanded memory and expanded weights (W in Figure 2) are iid with zero mean and variance σ 2. This assumption is reasonable as it enables a fair comparison of expanding memory and expanding hidden states. Finally, we assume every variable in the learned memory slots, i.e., m jk, follows the same distribution (zero mean, variance σ 2). This assumption may not be true after the network is trained, but is useful for proving theorems. We compute how the original dimensions in the hidden state are changed if we expand RNN. We denote the expanded hidden states by h i−1 and h i for the two time steps. We denote the weights connecting from h i−1 to h i by W ∈ R D×d. We focus on the original D-dimensional space, denoted as h (s) i. The connection is shown in Figure 2a. We have DISPLAYFORM1 where FORMULA0 is due to the independence and zero-mean assumptions of every element in W and h i−1. FORMULA0 is due to the independence assumption between W and h i−1.Next, we compute the effect of expanding memory slots. DISPLAYFORM2 i is the RNN hidden state after memory expansion. ∆c def = c − c, where c and c are the attention content vectors before and after memory expansion, respectively, at the current time step. 4 W (c) is the weight matrix connecting attention content to RNN states. The connection is shown in Figure 2b. Reusing the of, we immediately obtain DISPLAYFORM3 where ∆c k is an element of the vector ∆c. To prove Equation, it remains to show that Var(∆c k) ≤ σ 2. We now analyze how attention is computed. Let α 1, · · ·, α N +M be the unnormalized attention weights over the N + M memory slots. We notice that α 1, · · ·, α N remain the same after memory expansion. Then, the original attention probability is given by α j = α j /(α 1 + · · · + α N) for j = 1, · · ·, N. After memory expansion, the attention probability becomes α j = α j /(α 1 + · · · + α N +M), illustrated in FIG7. We have 4 We omit the time step in the notation for simplicity. Unnormalized measure DISPLAYFORM0 where DISPLAYFORM1 By our assumption of total attention DISPLAYFORM2 Then, we have DISPLAYFORM3 Here, is due to the assumption that m jk is independent and zero-mean, and is due to the independence assumption between β j and m jk. To obtain, we notice that DISPLAYFORM4 2 ≤ 1, concluding our proof. Note: In the theorem (and in experiments), memory expansion and hidden state expansion are done such that the total number of model parameters remain the same. The condition DISPLAYFORM5 α i,j in our theorem requires that the total attention to existing memory slots is larger than to the progressively added slots. This is a reasonable assumption because: During training, attention is trained in an ad hoc fashion to align information, and thus some of α i,j for 1 ≤ j ≤ N might be learned so that it is larger than a random memory slot; and For a new domain, we do not add a huge number of slots, and thus N +M j=N +1 α i,j will not dominate. We follow the original MultiNLI paper to choose the base model and most of the settings: 300D RNN hidden states, 300D pretrained GloVe embeddings for initialization, batch size of 32, and the Adam optimizer for training. The initial learning rate for Adam is tuned over the set {0.3, 0.03, 0.003, 0.0003, 0.00003}. It is set to 0.0003 based on validation performance. For the memory part, we set each slot to be 300D, which is the same as the RNN and embedding size. We tune the number of progressive memory slots in FIG9, which shows the validation performance on the source (Fic) and target (Gov) domains. We see that the performance is close to fine-tuning alone if only one memory slot is added. It improves quickly between 1 and 200 slots, and tapers off around 500. We thus choose to add 500 slots for each domain. Table 3 shows the dynamics of IDA with our progressive memory network. Comparing the upper-triangular values (in gray, showing out-of-domain performance) with diagonal values, we see that our approach can be quickly adapted to the new domain in an incremental fashion. Comparing lower-triangular values with the diagonal, we see that our approach does not suffer from the catastrophic forgetting problem as the performance of previous domains is gradually increasing if trained with more domains. After all data are observed, our model achieves the best performance in most domains (last row in Table 3), despite the incremental nature of our approach. We evaluate our approach on the task of dialogue response generation. Given an input text sequence, the task is to generate an appropriate output text sequence as a response in human-computer dialogue. For the target domain, we manually construct a very small dataset to mimic the scenario where quick adaptation has to be done to a new domain with little training data. In particular, we choose a random subset of 15k messageresponse pairs from the Ubuntu Dialog Corpus , a dataset of conversations about Ubuntu. We use a 9k-3k-3k data split. The base model is a sequence-to-sequence (Seq2Seq) neural network Following previous work, we use BLEU-2 and average Word2Vec embedding similarity (W2V-Sim) (; a) as the evaluation metrics. BLEU-2 is the geometric mean of unigram and bigram word precision penalized by length, and correlates with human satisfaction to some extent . W2V-Sim is defined as the cosine similarity between the averaged Word2Vec embeddings of the model outputs and the ground truths. Intuitively, BLEU measures hard word-level overlap between two sequences, whereas W2V-Sim measures soft similarity in a distributed semantic space. The for dialogue response generation are shown in Table 4. We see that BLEU-2 and W2V similarity are not necessarily consistent. For example, the memory-augmented RNN trained solely on source achieves the best source BLEU-2, whereas the proposed progressive memory has the highest W2V cosine similarity on S. However, our model variants (either expanding the vocabulary or not) achieve the best performance on most metrics (Lines 7 and 8). Moreover, it consistently outperforms all other IDA approaches. Following Experiment I, we conduct statistical test compared with Line 8. The test shows that our method is significantly better than the other IDA methods. you should be able to install the grub cd at the drive TAB9. Sample outputs of our IDA model S→T (F+M+V) from TAB9. In general, the evaluation of dialogue systems is noisy due to the lack of appropriate metrics . Nevertheless, our experiment provides additional evidence of the effectiveness of our approach. It also highlights our model's viability for both classification and generation tasks.

Represent the short summary for finding the corresponding scientific article (long)

We present a neural memory-based architecture for incremental domain adaptation, and provide theoretical and empirical results.

Represent the TLDR for finding the corresponding scientific article:

We present a new approach, SNIP, that is simple, versatile and interpretable; it prunes irrelevant connections for a given task at single-shot prior to training and is applicable to a variety of neural network models without modifications.


Question: represent the input:

Question: In compressed sensing, a primary problem to solve is to reconstruct a high dimensional sparse signal from a small number of observations. In this work, we develop a new sparse signal recovery algorithm using reinforcement learning (RL) and Monte CarloTree Search (MCTS). Similarly to orthogonal matching pursuit (OMP), our RL+MCTS algorithm chooses the support of the signal sequentially. The key novelty is that the proposed algorithm learns how to choose the next support as opposed to following a pre-designed rule as in OMP. Empirical are provided to demonstrate the superior performance of the proposed RL+MCTS algorithm over existing sparse signal recovery algorithms. We consider the compressed sensing (CS) problem [1; 2; 3], where for a given matrix A ∈ R m×n, m n, and a (noiseless) observation vector y = Ax 0, we want to recover a k-sparse vector/signal x 0 (k < m). Formally, it can be formulated as: subject to Ax = Ax 0 Related work There is a large collection of algorithms for solving the CS problem. Some foundational and classic algorithms include convex relaxation, matching and subspace pursuit [4; 5; 6] and iterative thresholding [7; 8]. In particular, two well-established methods are (i) Orthogonal Matching Pursuit (OMP) and (ii) Basis Pursuit (BP). OMP recovers x 0 by choosing the columns of A iteratively until we choose k columns. BP recovers x 0 by solving min Ax=y ||x|| 1. Because OMP and BP are extremely well studied theoretically [1; 2] and empirically, we use these two algorithms as the main baseline methods to compare against when evaluating the proposed RL+MCTS algorithm. Recent advancements in machine learning have opened a new frontier for signal recovery algorithms. Specifically, these algorithms take a deep learning approach to CS and the related error correction problem. The works in,, and apply ANNs and RNNs for encoding and/or decoding of signals x 0. Modern generative models such as Autoencoder, Variational Autoencoder, and Generative Adversarial Networks have also been used to tackle the CS problem with promising theoretical and empirical [15; 16; 17]. These works involve using generative models for encoding structured signals, as well as for designing the measurement matrix A. Notably, the empirical in these works typically use structured signals in x 0. For example, in and, MNIST digits and celebrity images are used for training and testing. Our contribution Differently from the above learning-based works, our innovation with machine learning is on signal recovery algorithms (as opposed to signal encoding or measurement matrix design). We do not assume the signals to be structured (such as images), but cope with general sparse signals. This underlying model for x 0 is motivated by the same assumptions in the seminal work on universal phase transitions by Donoho and Tanner in. Moreover, we assume the measurement matrix A is given. Extending to varying matrices A is left for future investigation. In this work, we approach the signal recovery problem using reinforcement learning (RL). Specifically, we leverage the Monte Carlo Tree Search (MCTS) technique with RL, which was shown to achieve outstanding performance in the game of Go [18; 19]. We further introduce special techniques to reduce the computational complexity for dealing with higher signal sparsity in CS. Experimental show that the proposed RL+MCTS algorithm significantly outperforms OMP and BP for matrix A of various sizes. In this section, we formulate the sparse signal recovery problem as a special sequential decision making problem, which we will solve using RL and MCTS. In the context of compressed sensing, a key challenge is to correctly choose the columns of A, or equivalently, the support of x 0, such that the problem is solved. To address this problem, we formulate it as a sequential decision making problem: an agent sequentially chooses one column of A at a time until it selects up to k columns such that the constraint in holds and the 0 -loss in is minimized. The MDP for compressed sensing can then be defined as follows. A state s ∈ S is a pair (y, S), where y is the observed signal generated according to x 0, and S ⊆ [n] is the set of the already selected columns of A, where [n] {1, . . ., n}. In our current setup, we assume the matrix A is fixed, so a state is not dependent on the sensing matrix. Terminal states are states s = (y, S) which satisfy one or more of the following conditions: (i) |S| = k (the maximum possible signal sparsity), or (ii) ||A S x s − y|| 2 2 < for some pre-determined. Here, A S stands for the submatrix of A that is constructed by the columns of A indexed by the set S, and x s is the optimal solution given that the signal support is S, For the action space, the set of all feasible actions at state Note that in compressed sensing, when an action a is taken (i.e., a new column of A is selected) for a particular state s = (y, S), the next state s is determined; that is, the MDP transition is deterministic. Finally, we define our reward function R: where α, γ > 0 are fixed hyperparameters, and x s is determined by. Different from existing compressed sensing algorithms, we propose to learn, via RL and MCTS, a policy to sequentially select the columns of A and reconstruct the sparse signal x 0, based on data generated for training. We generate the training data by generating k-sparse signals x 0 and computing the corresponding vectors y = Ax 0 (each k is randomly generated from 1 to m). For each signal y, we then use a "policy network" (to be explained in details later) along with MCTS to choose columns sequentially until k columns have been chosen. The traversed states will be used as our new training data for updating the policy network. Such a strategy allows us to move as much of the computational complexity as possible in testing (i.e., performing the sparse signal recovery task) into training, which shares a similar spirit to the work in. 3 The RL+MCTS Algorithm To learn a policy in the above sequential decision making formulation of CS, we employ a single neural network f θ to jointly model the policy π θ (a|s) and the state-value function V θ (s), where θ is the model parameter (i.e., the weights in a neural network). The policy π θ (a|s) defines a probability over all actions for a given state s, where the action set includes the possible next columns of A to pick and a stopping action. The value V θ (s) defines the long-term reward that an agent receives when we start from the state s and follow the given policy. We design two sets of input features for the policy/value network. The first set of input features is x s extended to a vector in R n with zeros in components whose indices are not in s. The second set of features is motivated by OMP, which is given by λ s:= A T (y − A S x s) ∈ R n, where y − A S x s is the residual vector associated with the solution x s. For the root state r in which no columns are chosen, x r is set to be the n-dimensional zero vector, and λ r:= A T y. Note that the OMP rule is exactly choosing the next column index whose corresponding component in |λ s | is the largest, where | · | is the absolute value taken component wise. The goal of the RL+MCTS algorithm is to iteratively train the policy network f θ. The high-level training structure is given in Algorithm 1. Algorithm 1 High-Level Training Procedure 1: initialize: j = 0, θ = θ0, θ0 random, fixed matrix A ∈ R m×n 2: while j < i (where i is a hyperparameter) do 3: 1) generate training samples from each (y, x0) pair by building a tree using Monte Carlo Tree Search (MCTS) and current f θ 2) train/update neural network parameters to getθ using the training samples from step 1. θ ←θ 6: Most of the details arise in step 1) of Algorithm 1. Similar to the AlphaGo Zero algorithm, the proposed RL+MCTS algorithm uses Monte Carlo Tree Search (MCTS) as a policy improvement operator to iteratively improve the policy network in the training phase. For a randomly generated pair (y, x 0), we use MCTS and the current f θ to generate new training samples to feed back into the neural network. We note that in the testing phase, MCTS can also be combined with the policy/value network to further boost the performance. Specifically, for each given observation vector y and the desired sparsity k, we run MCTS simulations multiple times to construct a search tree [21; 22; 23]. When training the proposed RL+MCTS algorithm, we employ the following technique for reducing the training complexity. First, we remark that using MCTS as a policy improvement operator can potentially be computationally expensive for relatively large matrix A (depending on the available computation resources). To address this challenge, we fix the maximum depth d of the MCTS tree; that is, we build the MCTS tree until we reach a depth of d. From then on, we roll-out the remaining levels of the tree by simply using the OMP rule to select all remaining columns until a total of k columns are chosen. This technique will be evaluated in the experiments in the next section. In this section, we present experimental for evaluating our proposed RL+MCTS algorithm and comparing it against two baseline methods: (i) OMP and (ii) BP (i.e., 1 minimization). We first present on the proposed RL+MCTS algorithm without limiting the tree depth. In this setting, we will be training and testing on matrices of size 7 × 15 and 15 × 50. The training parameters we use in our experiment is given in Table 2 We next show the using the RL+MCTS algorithm with reduced complexity as described in Section 3.3. Specifically, in a single MCTS search, we expand the tree to depth d, and then proceed to follow the OMP rule until a terminal state is reached. We now show the experiment for this version of the RL+MCTS algorithm. Specifically, we consider the 10 × 100 matrix in our evaluation. The training details of this experiment can be found in Table 2 in Appendix A. We train two models. A) We train a policy value network using the vanilla RL+MCTS algorithm without tree depth constraint. We train a policy value network by limiting the tree depth d = 6, which leads to a 40% reduction in training time per sample. Next, we first test the policy/value network trained from A) above. This policy/value network will select each column without MCTS. We then test the policy/value network trained from B) above: First, we test the policy/value network to pick the first column; For all subsequent columns up to k, we invoke the OMP rule. This is equivalent to setting the tree depth during testing to d = 2 and with no MCTS (M = 0). Using the same policy/value network, we also conduct an experiment where d = 6 and MCTS simulations is set to 1500 during testing. From Figure 1 (c), note that the vanilla RL+MCTS policy π θ (a|s) still performs slightly better than both OMP and BP. We see that training the RL+MCTS algorithm with a fixed tree depth gives us favorable versus OMP, vanilla RL+MCTS policy π θ (a|s), and BP. Average Prediction Times In Table 1, we give the average prediction times per signal in seconds. For OMP and BP, we use python libraries sklearn and cvx respectively. To illustrate the speed during testing, we measure the prediction times on a much less powerful machine than what was used during training. While training was accomplished on a i7 4790 (3.6 GHz) with a single GTX 780, the testing speeds in Table 2 were conducted on a Macbook Air with an Intel i5 clocked at 1.4 GHz and an integrated Intel HD 5000. We predict that the testing speeds can be greatly improved with a more powerful machine and further optimization in the source code. In general, we see that using just the policy/value network for prediction is in general slower than OMP, but on par with or better than BP. We have shown that the proposed RL+MCTS algorithm is a highly effective sparse signal decoder for the compressed sensing problem assuming no signal structure other than sparsity. Even without using MCTS in testing, the RL+MCTS algorithm's performance exceeds that of existing sparse signal recovery algorithms such as OMP and BP. The flexibility in the RL+MCTS algorithm's design further offers many interesting avenues for future research. For one, it is possible that the features chosen in our model can be further improved. Secondly, since the true signal x 0 is known in training, one may be able to leverage the information about x 0 to increase training sample efficiency. The training hyper-parameters may also be further tuned to improve performance. Broader settings of problems such as noisy observations and varying observation matrices A are under active investigation. In this appendix, we include the hyper-parameters of our experiments -see Table 2.

represent the short summary for finding the corresponding scientific article (long).

Formulating sparse signal recovery as a sequential decision making problem, we develop a method based on RL and MCTS that learns a policy to discover the support of the sparse signal.

Represent the short summary for finding the corresponding scientific article (long)

A backward model of previous (state, action) given the next state, i.e. P(s_t, a_t | s_{t+1}), can be used to simulate additional trajectories terminating at states of interest! Improves RL learning efficiency.