Question: Represent the scientific article to find a short TLDR

Question: The fields of artificial intelligence and neuroscience have a long history of fertile bi-directional interactions. On the one hand, important inspiration for the development of artificial intelligence systems has come from the study of natural systems of intelligence, the mammalian neocortex in particular. On the other, important inspiration for models and theories of the brain have emerged from artificial intelligence research. A central question at the intersection of these two areas is concerned with the processes by which neocortex learns, and the extent to which they are analogous to the back-propagation training algorithm of deep networks. Matching the data efficiency, transfer and generalisation properties of neocortical learning remains an area of active research in the field of deep learning. Recent advances in our understanding of neuronal, synaptic and dendritic physiology of the neocortex suggest new approaches for unsupervised representation learning, perhaps through a new class of objective functions, which could act alongside or in lieu of back-propagation. Such local learning rules have implicit rather than explicit objectives with respect to the training data, facilitating domain adaptation and generalisation. Incorporating them into deep networks for representation learning could better leverage unlabelled datasets to offer significant improvements in data efficiency of downstream supervised readout learning, and reduce susceptibility to adversarial perturbations, at the cost of a more restricted domain of applicability. the collection of areas responsible for visual object recognition, computes hierarchically organized representations much like state-of-the art convolutional neural networks (CNNs) optimized for the task . While there are impressive similarities in the learned representations between the ventral stream and CNNs, there are important differences in how those representations are learned. While CNNs are trained in a supervised manner using a gradient descent optimization algorithm with an explicit global objective on large labelled datasets, the ventral stream learns from a much larger dataset (visual experience) but with only very sparse labelling. The latter property of cortical learning is attractive to emulate in CNNs, and more broadly across deep learning models. Attractive, not only because of the ability to make use of unlabelled data during learning, but also because it will impart the models with superior generalization and transfer properties, as discussed below. The monkey's paw effect: the problem with specifying what without specifying how A well known and often encountered pitfall of numerical optimization algorithms for high dimensional problems, such as evolutionary algorithms, simulated annealing and also gradient descent, is that they regularly yield solutions matching what your objective specifies to the letter, but far from how you intended . The short story "The Monkey's Paw" by W. W. Jacobs provides a compelling metaphor. In that story, the new owner of a magical mummified monkey's paw of Indian origin is granted three wishes. The owner first wishes for $200, and his wish is eventually granted to the penny, but with the grave side effect that it is granted through a goodwill payment from his son's employer in response to his untimely death in a terrible machinery accident . The Monkey's Paw effect is also applicable to gradient descent-based optimization of deep neural nets. The relative data-hungriness of current supervised learning strategies, and the use of data augmentation to improve generalization reflect the precarious position we are in of needing to micromanage the learning processes. Adversarial examples are evidence that the monkey's paw effect none-the-less persists. It is temping to continue with the current paradigm and re-inject adversarial examples back into the learning data stream. Extrapolating, this goes in the direction of specifying the negative space of the objective, all those things the optimization should not do to solve the problem, which is potentially infinite, and rather risky in production environments like self-driving cars. Adversarial examples represent an opportunity to address the issue in a more fundamental way . It has been argued by that if we could design deep learning systems with the explicit objective of "disentangling the underlying factors of variation" in an unsupervised manner, then there is much to be gained for generalization and transfer. Such an approach offers a promising solution to the Monkey's Paw effect, as there is an explicit objective of learning good representations, from which generalization and transfer follow by definition. 1 One small challenge remains: how to express the objective of learning good representations? If we restrict ourselves to the subset of all possible inputs for which the neocortex learns good representations, the local processes of synaptic plasticity may provide valuable clues. The neocognitron model , the original CNN architecture, learned visual features through self-organization using local rules. Since its conception, our understanding of the neocortex and its neurons and synapses has progressed considerably. Recent insights into the local plasticity rules for learning in the neocortex offer new inspiration for deep representation learning paradigms that learn "disentangled representations" from large unlabelled datasets in an unsupervised manner. A selection of recent insights into the systems of plasticity of the neocortex is shown in Fig. 1. A new dendrite-centric view of synaptic plasticity is emerging with the discovery of the NMDA spike, a non-linear mechanism hypothesized to associate co-activated synapses through potentiation or structural changes driven by the ing calcium currents (; ;) (Fig. 1A-B). Such associations, in the form of co-coding clusters of synapses, have recently been experimentally observed using optical techniques (Fig. 1C). Moreover neurons in the neocortex are known to form small cliques of all-to-all connected neurons which drive co-coding , a process that would be self-reinforced through dendritic clustering by NMDA spikes (Fig. 1D). Martinotti neurons, which are activated by such cliques of pyramidal neurons, and subsequently inhibit pyramidal dendrites provide well-timed inhibition to block further NMDA spikes , and put a limit on the maximal pyramidal clique size, but also suppress activation of competing cliques (e.g. Winner-take-all (WTA) dynamics). Together, such plasticity mechanisms appear to form basic building blocks for representation learning in the feedforward pathway of the neocortex using local learning rules. While long known competitive strategies for unsupervised representation learning indeed rely on WTA dynamics , deep learning approaches incorporating these increasingly apparent dendritic dimensions of learning processes have yet to be proposed . Unlike CNNs, the neocortex also has a prominent feedback pathway down the hierarchy, whereby topdown input from upper layers innervate the apical tufts of pyramidal cells in layer 1 of a given cortical region . Associations between top-down and feed-forward (bottom-up) activation are known to trigger dendritic calcium spikes and dendritic bursting , which again specifically activates the WTA dynamics of the Martinotti neurons , but disinhibitory VIP neurons can also modulate their impact . These feed-back pathways have been proposed to implement predictive coding , and error back-propagation for supervised learning algorithms . While their importance for rapid object recognition has been recently demonstrated, their computational role remained inconclusive . With the demonstrated applicability of supervised learning for a broad range of problems and data distributions, and an ever expanding toolbox of optimized software libraries, it is unlikely that supervised learning, back-propagation and gradient descent will be dethroned as the work horses of AI for many years to come. Nonetheless, as applications of deep networks are moving into regions where sparse data, generalization and transfer are increasingly important, unsupervised approaches designed with the explicit goal of learning good representations from mere observation may find an important place in the AI ecosystem. Quoting Yann LeCun 2 "If intelligence is a cake, the bulk of the cake is unsupervised learning, the icing on the cake is supervised learning, and the cherry on the cake is reinforcement learning." A promising strategy would be to assume learning with sparse labels, overcoming adversarial examples, transfer learning, and few-shot learning together as the success criteria for the further development of the powerful unsupervised approaches we seek. Recent advances in our understanding of the processes of neocortical plasticity may well offer useful inspiration, but let's close with some words of moderation. Biology's solutions also show us there will be no free lunch, i.e. neocortical unsupervised learning algorithms will be less general than supervised learning by gradient descent. Neocortex relies on structure at specific spatial and temporal scales in its input streams to learn representations. Evolution has had millions of years to configure the sensory organs to provide signals to the neocortex in ways that it can make sense of them, and that serve the animal's ecological niche. We should not expect, for example, cortical unsupervised learning algorithms to cluster frozen white noise images. A neocortical solution requires a neocortical problem (e.g. from the so-called "Brain set" ), so if we are to successfully take inspiration from it, we must also work within its limitations.

Represent the short summary for finding the corresponding scientific article (long):

Inspiration from local dendritic processes of neocortical learning to make unsupervised learning great again.

Represent the TLDR for finding the corresponding scientific article

Our paper analyses the tremendous representational power of networks especially with 'skip connections', which may be used as a method  for better generalization.


Question: Represent the article for finding a single-sentence TLDR

Question: Real-world Question Answering (QA) tasks consist of thousands of words that often represent many facts and entities. Existing models based on LSTMs require a large number of parameters to support external memory and do not generalize well for long sequence inputs. Memory networks attempt to address these limitations by storing information to an external memory module but must examine all inputs in the memory. Hence, for longer sequence inputs the intermediate memory components proportionally scale in size ing in poor inference times and high computation costs. In this paper, we present Adaptive Memory Networks (AMN) that process input question pairs to dynamically construct a network architecture optimized for lower inference times. During inference, AMN parses input text into entities within different memory slots. However, distinct from previous approaches, AMN is a dynamic network architecture that creates variable numbers of memory banks weighted by question relevance. Thus, the decoder can select a variable number of memory banks to construct an answer using fewer banks, creating a runtime trade-off between accuracy and speed. AMN is enabled by first, a novel bank controller that makes discrete decisions with high accuracy and second, the capabilities of a dynamic framework (such as PyTorch) that allow for dynamic network sizing and efficient variable mini-batching. In our , we demonstrate that our model learns to construct a varying number of memory banks based on task complexity and achieves faster inference times for standard bAbI tasks, and modified bAbI tasks. We achieve state of the art accuracy over these tasks with an average 48% lower entities are examined during inference. Question Answering (QA) tasks are gaining significance due to their widespread applicability to recent commercial applications such as chatbots, voice assistants and even medical diagnosis BID7 ). Furthermore, many existing natural language tasks can also be re-phrased as QA tasks. Providing faster inference times for QA tasks is crucial. Consumer device based question-answer services have hard timeouts for answering questions. For example, Amazon Alexa, a popular QA voice assistant, allows developers to extend the QA capabilities by adding new "Skills" as remote services BID0 ). However, these service APIs are wrapped around hard-timeouts of 8 seconds which includes the time to transliterate the question to text on Amazon's servers and the round-trip transfer time of question and the answer from the remote service, and sending the response back to the device. Furthermore, developers are encouraged to provide a list of questions ("utterances") apriori at each processing step to assist QA processing BID0 ).Modeling QA tasks with LSTMs can be computationally expensive which is undesirable especially during inference. Memory networks, a class of deep networks with explicit addressable memory, have recently been used to achieve state of the art on many QA tasks. Unlike LSTMs, where the number of parameters grows exponentially with the size of memory, memory networks are comparably parameter efficient and can learn over longer input sequences. However, they often require accessing all intermediate memory to answer a question. Furthermore, using focus of attention over the intermediate state using a list of questions does not address this problem. Soft attention based models compute a softmax over all states and hard attention models are not differentiable and can be difficult to train over a large state space. Previous work on improving inference over memory networks has focused on using unsupervised clustering methods to reduce the search space BID2; BID19 ). Here, the memory importance is not learned and the performance of nearest-neighbor style algorithms is often comparable to a softmax operation over memories. To provide faster inference for long sequence-based inputs, we present Adaptive Memory Networks (AMN), that constructs a memory network on-the-fly based on the input. Like past approaches to addressing external memory, AMN constructs the memory nodes dynamically. However, distinct from past approaches, AMN constructs a memory architecture with network properties that are decided dynamically based on the input story. Given a list of possible questions, our model computes and stores the entities from the input story in a memory bank. The entities represent the hidden state of each word in the story while a memory bank is a collection of entities that are similar w.r.t the question. As the number of entities grow, our network learns to construct new memory banks and copies entities that are more relevant towards a single bank. Entities may reside in different bank depending on their distance from the question. Hence, by limiting the decoding step to a dynamic number of constructed memory banks, AMN achieves lower inference times. AMN is an end-to-end trained model with dynamic learned parameters for memory bank creation and movement of entities. Figure 1 demonstrates a simple QA task where AMN constructs two memory banks based on the input. During inference only the entities in the left bank are considered reducing inference times. To realize its goals, AMN introduces a novel bank controller that uses reparameterization trick to make discrete decisions with high accuracy while maintaining differentiability. Finally, AMN also models sentence structures on-the-fly and propagates update information for all entities that allows it to solve all 20 bAbI tasks. Memory Networks: Memory networks store the entire input sequence in memory and perform a softmax over hidden states to update the controller BID27; BID23 ). DMN+ connects memory to input tokens and updates them sequentially BID29 ). For inputs that consist of large number of tokens or entities, these methods can be expensive during inference. AMN stores entities with tied weights in different memory banks. By controlling the number of memory banks, AMN achieves low inference times with reasonable accuracy. Nearest neighbor methods have also been explored over memory networks. For example, Hierarchical Memory Networks separates the input memory into groups using the MIPS algorithm BID2 ). However, using MIPS is as slow as a softmax operation, so the authors propose using an approximate MIPS that gives inferior performance. In contrast, AMN is end to end differentiable, and reasons which entities are important and constructs a network with dynamic depth. Neural Turing Machine (NTM) consists of a memory bank and a differentiable controller that learns to read and write to specific locations BID9 ). In contrast to NTMs, AMN memory bank controller is more coarse grained and the network learns to store entities in memory banks instead of specific locations. AMN uses a discrete bank controller that gives improved performance for bank controller actions over NTM's mechanisms. However, like NTMs, our design is consistent with the modeling studies of working memory by BID11 ) where the brain performs robust memory maintenance and may maintain multiple working representations for individual working tasks. Sparse access memory uses approximate nearest neighbors (ANN) to reduce memory usage in NTMs BID19 ). However, ANNs are not differentiable. AMN, uses a input specific memory organization that does not create sparse structures. This limits access during inference to specific entities reducing inference times. Graph-based networks, (GG-NNs, BID16 and) use nodes with tied weights that are updated based on gated-graph state updates with shared weights over edges. However, unlike AMN, they require strong supervision over the input and teacher forcing to learn the graph structure. Furthermore, the cost of building and training these models is expensive and if every edge is considered at every time-step the amount of computation grows at the order of O(N 3) where N represents the number of nodes/entities. AMN does not use strong supervision but can solve tasks that require transitive logic by modeling sentence walks on the fly. EntNet constructs dynamic networks based on entities with tied weights for each entity BID12 ). A key-value update system allows it to update relevant (learned) entities. However, Entnet uses soft-attention during inference to attend to all entities that incur high inference costs. To summarize, majority of the past work on memory networks uses softmax over memory nodes, where each node may represent input or an entity. In contrast, AMN learns to organize memory into various memory banks and performs decode over fewer entities reducing inference times. Conditional Computation & Efficient Inference: AMN is also related to the work on conditional computation which allows part of networks to be active during inference improving computational efficiency BID1 ). Recently, this has been often accomplished using a gated mixture of experts BID4; BID22 ). AMN conditionally attends to entities in initial banks during inference improving performance. For faster inference using CNNs, pruning BID15; BID10 ), low rank approximations BID3 ), quantization and binarization BID20 ) and other tricks to improve GEMM performance BID25 ) have been explored. For sequence based inputs, pruning and compression has been explored BID6; BID21 ). However, compression in irregular sparsity that reduces memory costs but may not reduce computation costs. Adaptive computation time BID8 ) learns the number of steps required for inferring the output and this can also be used to reduce inference times BID5 ). AMN uses memory networks with dynamic number of banks to reduce computation costs. Dynamic networks: Dynamic neural networks that change structure during inference have recently been possible due to newer frameworks such as Dynet and PyTorch. Existing work on pruning can be implemented using these frameworks to reduce inference times dynamically like dynamic deep network demonstrates BID17 ). AMN utilizes the dynamic architecture abilities to construct an input dependent memory network of variable memory bank depth and the dynamic batching feature to process a variable number of entities. Furthermore, unlike past work that requires an apriori number of fixed memory slots, AMN constructs them on-the-fly based on the input. The learnable discrete decision-making process can be extended to other dynamic networks which often rely on REINFORCE to make such decisions BID17 ).Neuroscience: Our network construction is inspired by work on working memory representations. There is sufficient evidence for multiple, working memory representations in the human brain (Hazy et al. FORMULA0). Semantic memory BID24 ), describes a hierarchical organization starting with relevant facts at the lowest level and progressively more complex and distant concepts at higher levels. AMN constructs entities from the input stories and stores the most relevant entities based on the question in the lowest level memory bank. Progressively higher level memory banks represent distant concepts (and not necessarily higher level concepts for AMN). Other work demonstrates organization of human memory in terms of "priority structure" where attention is a gate-keeper of working memory-guided by executive control's goals, plans, and intentions as in BID26, similar in spirit to AMN's question guided network construction. In this section, we describe the design process and motivation of our memory module. Our memory network architecture is created during inference time for every story. The architecture consists of different memory banks and each memory bank stores entities from the input story. Hence, a memory entity represents the hidden state of each entity (each word in our case) from the input story while a memory bank is a collection of entities. Intuitively, each memory bank stores entities that have a similar distance score from the question. At a high level, entities are gradually and recurrently copied through memory banks to filter out irrelevant nodes such that in the final inference stage, fewer entities are considered by the decoder. Note that the word filter implies a discrete decision and that recurrence implies time. If we were to perform a strict cut off and remove entities that appear to be irrelevant at each time step, learning the reasoning logic that requires previous entities that were cut off would not be possible. Thus, smoothed discretization is required. We design filtering to be a two-stage pseudo-continuous process to simulate discrete cut offs (Π move, Π new), while keeping reference history. The overall memory (M) consists of multiple memory banks. A memory bank is a collection or group of entities (m 0...l), where m 0 denotes the initial and most general bank and m l denotes the most relevant bank. Note that |l| is input dependent and learned. First, entities are moved from m 0 gradually towards m l based off of their individual relevance to the question and second, if m l becomes too saturated, m l+1 is created. Operations in the external memory allowing for such dynamic restructuring and entity updates are described below. Note that these operations still maintain end to end differentiability.1. Memory bank creation (Π new), which creates a new memory bank depending on the current states of entities m i. If the entropy, or information contained (explained below), of m i is too high, Π new (m i) will learn to create a new memory bank m i+1 to reduce entropy.2. Moving entities across banks (Π move), which determines which entities are relevant to the current question and move such entities to further (higher importance) memory banks.3. Adding/Updating entities in a bank (Π au), which adds entities that are not yet encountered to the first memory bank m 0 or if the entity is already in m 0, the operation updates the entity state.4. Propagating changes across entities (Π prop), which updates the entity states in memory banks based on node current states Π prop (M) and their semantic relationships. This is to communicate transitive logic. Both Π new, Π move require a discrete decision (refer to section 4.2.1.), and in particular, for Π new we introduce the notion of entropy. That is to say if m i contains too many nodes (the entropy becomes too high), the memory module will learn to create a new bank m i+1 and move nodes to m i+1 to reduce entropy. By creating more memory banks, the model spreads out the concentration of information which in turn better discretizes nodes according to relevance. A high-level overview is shown in FIG1, followed by a mathematical detail of the model's modules. Our model adopts the encoder-decoder framework with an augmented adaptive memory module. For an overview of the algorithm, refer to Section A.1.Notation and Problem Statement: Given a story represented by N input sentences (or statements), i.e., (l 1, · · ·, l N), and a question q, our goal is to generate an answer a. Each sentence l is a sequence of N words, denoted as (w 1, · · ·, w Ns), and a question is a sequence of N q words denoted as (w 1, · · ·, w Nq). Throughout the model we refer to entities; these can be interpreted as a 3-tuple of e w = (word ID wi, hidden state w, question relevance strength s). Scalars, vectors, matrices, and dot products are denoted by lower-case letters, boldface lower-case letters and boldface capital letters, and angled brackets respectively. The input to the model, starting with the encoder, are story-question input pairs. On a macro level, sentences l 1... N are processed. On a micro level, words w 1... Ns are processed within sentences. For each w i ∈ l i, the encoder maps w i to a hidden representation and a question relevance strength ∈. The word ID of w i is passed through a standard embedding layer and then encoded through an accumulation GRU. The accumulation GRU captures the entity states through time by adding the output of each GRU time step to its respective word, stored in a lookup matrix. The initial states of e w are set to this GRU output. Meanwhile, the question is also embedded and encoded in the same manner sans accumulation. In the following, the subscripts i, j are used to iterate through the total number of words in a statement and question respectively, D stores the accumulation GRU output, and w i is a GRU encoding output. The last output of the GRU will be referred to as w N, w Nq for statements and questions. DISPLAYFORM0 DISPLAYFORM1 To compute the question relevance strength s ∈ for each word, the model uses GRU-like equations. The node strengths are first initialized to Xavier normal and the inputs are the current word states w in, the question state w Nq, and when applicable, the previous strength. Sentences are processed each time step t. DISPLAYFORM2 DISPLAYFORM3 DISPLAYFORM4 In particular, equation FORMULA2 shows where the model learns to lower the strengths of nodes that are not related the question. First, a dot product between the current word states and question state are computed for similarity (high correlation), then it is subtracted from a 1 to obtain the dissimilarity. We refer to these operations as SGRU (Strength GRU) in Algorithm 1. The adaptive memory module recurrently restructures entities in a question relevant manner so the decoder can then consider fewer entities (namely, the question relevant entities) to generate an answer. The following operations are performed once per sentence. As mentioned earlier, discrete decisions are difficult for neural networks to learn so we designed a specific memory bank controller Π ctrl for binary decision making. The model takes ideas from the reparameterization trick and uses custom backpropagation to maintain differentiability. In particular, the adaptive memory module needs to make two discrete decisions on a {0, 1} basis, one in Π new to create a new memory bank and the other in Π move to move nodes to a different memory bank. The model uses a scalar p ∈ {0, 1} to parameterize a Bernoulli distribution where the realization H, is the decision the model makes. However, backpropagation through a random node is intractable, so the model detaches H from the computation graph and introduces H as a new node. Finally, H is used as a mask to zero out entities in the discrete decision. Meanwhile, p is kept in the computation graph and has a special computed loss (Section 4.4). The operations below will be denoted as Π ctrl and has two instances: one for memory bank creation Π new and one for moving entities across banks Π move. In equation 9, depending on what Π ctrl is used for, q is a polymorphic function and will take on a different operation and * will be a different input. Examples of such are given in their respective sections (4.2.2.1, 4.2.2.2). DISPLAYFORM0 4.2.2 MEMORY BANK OPERATIONS 1. Memory bank creation Π new: To determine when a new memory bank is created, in other words, if the current memory bank becomes too saturated, the memory bank controller (4.2.1.) will make a discrete decision to create a new memory bank. Here, q (eq 9) is a fully connected layer and the input is the concatenation of all the current memory bank m i's entity states [w 0 ...w i] ∈ R 1,n|ew|. Intuitively, q will learn a continuous decision that is later discretized by eq 10 based on entity states and the number of entities. Note this is only performed for the last memory bank. DISPLAYFORM1 2. Moving entities through memory banks: Similar to Π new, individual entities' relevance scores are passed into the bank controller to determine H as the input. The relevance score is computed by multiplying an entity state by its respective relevance ∈ R n,|ew|. Here, q has a slight modification and is the identity function. Note that this operation can only be performed if there is a memory bank to move nodes to, namely if m i+1 exists. Additionally, each bank has a set property where it cannot contain duplicate nodes, but the same node can exist in two different memory banks. DISPLAYFORM2 3. Adding/Updating entities in a bank: Recall that entities are initially set to the output of D. However, as additional sentences are processed, new entities and their hidden states are observed. In the case of a new entity e w, the entity is added to the first memory bank m 0. If the entity already exists in m 0, then e w's corresponding hidden state is updated through a GRU. This procedure is done for all memory banks. DISPLAYFORM3 4. Propagating updates to related entities: So far, entities exist as a bag of words model and the sentence structure is not maintained. This can make it difficult to solve tasks that require transitive reasoning over multiple entities. To track sentence structure information, we model semantic relationships as a directed graph stored in adjacency matrix A. As sentences are processed word by word, a directed graph is drawn progressively from w 0... w i...w N. If sentence l k's path contains nodes already in the current directed graph, l k will include said nodes in its path. After l k is added to A, the model propagates the new update hidden state information a i among all node states using a GRU. a i for each node i is equal to the sum of the incoming edges' node hidden states. Additionally, we add a particular emphasis on l k to simulate recency. At face value, one propagation step of A will only have a reachability of its immediate neighbor, so to reach all nodes, A is raised to a consecutive power r to reach and update each intermediate node. r can be either the longest path in A or a set parameter. Again, this is done within a memory bank for all memory banks. For entities that have migrated to another bank, the update for these entities is a no-op but propagation information as per the sentence structure is maintained. A single iteration is shown below: DISPLAYFORM4 When nodes are transferred across banks, A is still preserved. If intermediate nodes are removed from a path, a transitive closure is drawn if possible. After these steps are finished at the end of a sentence, namely, the memory unit has reasoned through how large (number of memory banks) the memory should be and which entities are relevant at the current point in the story, all entities are passed through the strength modified GRU (4.1, eq 5-8) to recompute their question relevance (relevance score). After all sentences l 1...N are ingested, the decode portion of the network learns to interpret the from the memory banks. The network iterates through the memory banks using a standard attention mechanism. To force the network to understand the question importance weighting, the model uses an exponential function d to weight important memory banks higher. C m are the hidden states contained in memory m, s m are the relevance strengths of memory bank m, w Nq is the question hidden state, ps is the attention score, r, h are learned weight masks, g are the accumulated states, and l is the final logits prediction. During inference, fewer memory banks are considered. DISPLAYFORM0 Loss is comprised of two parts, answer loss, which is computed from the given annotations, and secondary loss (from Π new, Π move), which is computed from sentence and story features at each sentence time step l 0...N. Answer loss is standard cross entropy at the end of the story after l N is processed. DISPLAYFORM0 After each sentence l i, the node relevance s li is enforced by computing the expected relevance E[s li]. E[s] is determined by nodes that are connected to the answer node a in a directed graph; words that are connected to a are relevant to a. They are then weighted with a deterministic function of distance from a. DISPLAYFORM1 Additionally, bank creation is kept in check by constraining p li w.r.t. the expected number of memory banks. The expected number of memory banks can be thought of as a geometric distribution ∼ Geometric(p li) parameterized byp li, a hyperparameter. Typically, at each sentence stepp is raised to the inverse power of the current sentence step to reflect the amount of information ingested. Intuitively, this loss ensures banks are created when a memory bank contains too many nodes. On the other hand, the learned mask q (eq. 9) enables the model to weight certain nodes a higher entropy to prompt bank creation. Through these two dependencies, the model is able to simulate bank creation as a function of the number of nodes and the type of nodes in a given memory bank. DISPLAYFORM2 All components combined, the final loss is given in the following equation DISPLAYFORM3 In this section, we evaluate AMN accuracy and inference times on the bAbI dataset and extended bAbI tasks dataset. We compare our performance with Entnet BID12 ), which recently achieved state of the art on the bAbi dataset. For accuracy measurements, we also compare with DMN+ and encoder-decoder methods. Finally we discuss the time trade offs between AMN and current SOTA methods. The portion regarding inference times are not inclusive of story ingestion. We summarize our experiments as follows:• We are able to solve all bAbi tasks using AMN. Furthermore, AMN is able to reason important entities and propagate them to the final memory bank allowing for 48% fewer entities examined during inference.• We construct extended bAbI tasks to evaluate AMN behavior. First, we extend Task 1 for multiple questions in order to gauge performance in a more robust manner. For example, if a reasonable set of questions are asked (where reasonable means that collectively they do not require all entities to answer implying entities can be filtered out), will the model still sufficiently reason through entities. We find that our network is able to reason useful entities for both tasks and store them in the final memory bank. Furthermore, we also scale bAbI for a large number of entities and find that AMN provides additional benefits at scale since only relevant entities are stored in the final memory bank. We implement our network in PyTorch BID18 ). We initialize our model using Xavier initialization, and the word embeddings utilize random uniform initialization ranging from − √ 3 to √ 3. The learning rate is set as 0.001 initially and updated with a learning rate scheduler. E[s] contains nodes in the connected components of A containing the answer node a which has relevance scores sampled from a Gaussian distribution centered at 0.75 with a variance of 0.05 (capped at 1). Nodes that are not in the connected component containing a are similarly sampled from a Gaussian centered from 0.3 with a variance of 0.1 (capped at 0).p li is initially set to 0.8 and β varies depending on the story length from 0.1 ≤ β ≤ 0.25. Note that for transitive tasks,p li is set to 0.2. We train our models using the Adam optimizer BID14. The bAbI task suite consists of 20 reasoning tasks that include deduction, induction, path finding etc. Results are from the following parameters: ≤ 200 epochs, best of 10 runs. TAB1 shows the inference performance in terms of the number of entities examined. A task is considered passed if the error rate is less than 5%.We find that AMN creates 1 − 6 memory banks for different tasks. We also find that 8 tasks can be solved by looking at just one memory bank and 14 tasks can be solved with half the total number of memory banks. Lastly, all tasks can be solved by examining less than or equal the total number of entities (e ∈ M ≤ |V | +)1. Tasks that cannot be solved in fewer than half the memory banks either require additional entities due to transitive logic or have multiple questions. For transitive logic, additional banks could be required as an relevant nodes may be in a further bank. However, this still avoids scanning all banks. In the case of multiple questions, all nodes may become necessary to construct all answers. We provide additional evaluation in Appendix to examine memory bank behavior for certain tasks. TAB5 shows the number of banks created and required to solve a task, as well as the ratio of entities examined to solve the task. TAB2 shows the complexity of AMN and other SOTA models. Entnet uses an empirically selected parameter, typically set to the number of vocabulary words. GGT-NN uses the number of vocabulary words and creates new k new nodes intermittently per sentence step. For tasks where nodes are easily separable where nodes are clearly irrelevant to the question(s), AMN is able to successfully reduce the number of nodes examined. However for tasks that require more information, such as counting (Task 7), the model is still able to obtain the correct answer Table 2: Memory bank analysis of indicative tasks.without using all entities. Lastly, transitive logic tasks where information is difficult to separate due to dependencies of entities, the model creates very few banks (1 or 2) and uses all nodes to correctly generate an answer. We note that in the instance where the model only creates one bank, it is very sparse, containing only one or two entities. Because variations in computation times in text are minute, the number of entities required to construct an answer are of more interest as they directly correspond to the number of computations required. Additionally, due to various implementations of current models, their run times can significantly vary. However, for the comparison of inference times, AMN's decoder and EntNet's decoder are highly similar and contain roughly the same number of operations. We extend the bAbI tasks by adding additional entities and sentences and adding multiple questions for a single story, for Task 1. We increase the the number of entities to 100 entities in the task generation system instead of existing 38. We also extend the story length to 90 to ensure new entities are referenced. We find that AMN creates 6 memory banks and the ratio of entities in the final banks versus the overall entities drops to 0.13 given the excess entities that are not referenced in the questions. Multiple questions: We also augment the tasks with multiple questions to understand if AMN can handle when a story has multiple questions associated with it. We extend our model to handle multiple questions at once to limit re-generating the network for every question. To do so, we modify bAbi to generate several questions per story for tasks that do not currently have multiple questions. For single supporting fact (Task 1), the model creates 3 banks and requires 1 bank to successfully pass the task. Furthermore, the ratio of entities required to pass the task only increases by 0.16 for a total of 0.38. In this paper, we present Adaptive Memory Network that learns to adaptively organize the memory to answer questions with lower inference times. Unlike NTMs which learn to read and write at individual memory locations, Adaptive Memory Network demonstrates a novel design where the learned memory management is coarse-grained that is easier to train. Through our experiments, we demonstrate that AMN can learn to reason, construct, and sort memory banks based on relevance over the question set. AMN architecture is generic and can be extended to other types of tasks where the input sequence can be separated into different entities. In the future, we plan to evaluate AMN over such tasks to evaluate AMN generality. We also plan to experiment with larger scale datasets (beyond bAbI, such as a document with question pairs) that have a large number of entities to further explore scalability. Method Complexity Entnet BID12 We describe our overall algorithm in pseudo-code in this section. We follow the notation as described in the paper. DISPLAYFORM0 Algorithm 1 AMN(S, q, a) DISPLAYFORM1 for word w ∈ s do 4: DISPLAYFORM2 end for 6: DISPLAYFORM3 for memory bank m i ∈ M do 8: DISPLAYFORM4 n mi ← SGRU(D, n mi) We compare the computations costs during the decode operation during inference for solving the extended bAbi task. We compute the overheads for AMN Entnet BID12 ) and GGT-NN. TAB2 gives the decode comparisons between AMN, Entnet and GGT-NN. Here, |V | represents to the total number of entities for all networks. GGT-NN can dynamically create nodes and k k is hyper parameter the new nodes created for S sentences in input story. α is the percent of entities stored in the final bank w.r.t to the total entities for AMN.We compare the wall clock execution times for three tasks within bAbI for 1000 examples/task. We compare the wall-clock times for three tasks. We compare the inference times of considering all banks (and entities) versus the just looking at the passing banks as required by AMN. We find that AMN requires fewer banks and as a consequence fewer entities and saves inference times. In this section, we understand memory bank behavior of AMN. Figure 3 shows the memory banks and the entity creation for a single story example, for some of the tasks from bAbI. Depending upon the task, and distance from the question AMN creates variable number of memory banks. The heatmap demonstrates how entities are copied across memory banks. Grey blocks indicate absence of those banks. Under review as a conference paper at ICLR 2018 Figure 4 shows how propagation happens after every time step. The nodes represent entities corresponding to words in a sentence. As sentences are processed word by word, a directed graph is drawn progressively from w 0... w i...w N. If sentence l k's path contains nodes already in the current directed graph, l k will include said nodes in the its path. After l k is added to A, the model propagates the new update hidden state information a i among all node states using a GRU. a i for each node i is equal to the sum of the incoming edges' node hidden states. Additionally, we add a particular emphasis on l k to simulate recency. At face value, one propagation step of A will only have a reachability of its immediate neighbor, so to reach all nodes, A is raised to a consecutive power r to reach and update each intermediate node. r can be either the longest path in A or a set parameter.

represent the short summary for finding the corresponding scientific article (long):

We propose a new anytime neural network which allows partial evaluation by subnetworks with different widths as well as depths.


Question: represent the scientific article to find a short tldr

Question: When a bilingual student learns to solve word problems in math, we expect the student to be able to solve these problem in both languages the student is fluent in, even if the math lessons were only taught in one language. However, current representations in machine learning are language dependent. In this work, we present a method to decouple the language from the problem by learning language agnostic representations and therefore allowing training a model in one language and applying to a different one in a zero shot fashion. We learn these representations by taking inspiration from linguistics, specifically the Universal Grammar hypothesis and learn universal latent representations that are language agnostic . We demonstrate the capabilities of these representations by showing that the models trained on a single language using language agnostic representations achieve very similar accuracies in other languages. Anecdotally speaking, fluent bilingual speakers rarely face trouble translating a task learned in one language to another. For example, a bilingual speaker who is taught a math problem in English will trivially generalize to other known languages. Furthermore there is a large collection of evidence in linguistics arguing that although separate lexicons exist in multilingual speakers the core representations of concepts and theories are shared in memory BID1 BID28 BID6. The fundamental question we're interested in answering is on the learnability of these shared representations within a statistical framework. We approached this problem from a linguistics perspective. Languages have vastly varying syntactic features and rules. Linguistic Relativity studies the impact of these syntactic variations on the formations of concepts and theories BID5. Within this framework of study, the two schools of thoughts are linguistic determinism and weak linguistic influence. Linguistic determinism argues that language entirely forms the range of cognitive processes, including the creation of various concepts, but is generally agreed to be false BID18 BID5. Although there exists some weak linguistic influence, it is by no means fundamental BID0. The superfluous nature of syntactic variations across languages brings forward the argument of principles and parameters (PnP) which hypothesizes the existence of a small distributed parameter representation that captures the syntactic variance between languages denoted by parameters (e.g. head-first or head-final syntax), as well as common principles shared across all languages BID12. Universal Grammar (UG) is the study of principles and the parameters that are universal across languages BID29.The ability to learn these universalities would allow us to learn representations of language that are fundamentally agnostic of the specific language itself. Doing so would allow us to learn a task in one language and reap the benefits of all other languages without needing multilingual datasets. We take inspiration from the UG hypothesis and learn latent representations that are language agnostic which allow us to solve downstream problems in new languages without the need of any language-specific training data. We do not make any claims about the Universal Grammar hypothesis, but simply take inspiration from it. Our work attempts to unite universal (task agnostic) representations with multilingual (language agnostic) representations BID30 BID26. The recent trend in universal representations has been moving away from context-less unsupervised word embeddings to contextrich representations. Deep contextualized word representations (ELMo) trains an unsupervised language model on a large corpus of data and applies it to a large set of auxiliary tasks BID30. These unsupervised representations boosted the performance of models on a wide array of tasks. Along the same lines BID26 showed the power of using latent representations of translation models as features across other non-translation tasks. In general, initializing models with pre-trained language models shows promise against the standard initialization with word embeddings. Even further, BID31 show that an unsupervised language model trained on a large corpus will contain a neuron that strongly correlates with sentiment without ever training on a sentiment task implying that unsupervised language models maybe picking up informative and structured signals. In the field of multilingual representations, a fair bit of work has been done on multilingual word embeddings. BID2 explored the possibility of training massive amounts of word embeddings utilizing either parallel data or bilingual dictionaries via the SkipGram paradigm. Later on an unsupervised approach to multilingual word representations was proposed by BID9 which utilized an adversarial training regimen to place word embeddings into a shared latent space. Although word embeddings show great utility, they fall behind methods which exploit sentence structure as well as words. Less work has been done on multilingual sentence representations. Most notably both BID32 and BID4 propose a way to learn multilingual sentence representation through a translation task. We train downstream models using language agnostic universal representations on a set of tasks and show the ability for the downstream models to generalize to languages that we did not train on. Statistical language models approximate the probability distribution of a series of words by predicting the next word given a sequence of previous words. DISPLAYFORM0 where w i are indices representing words in an arbitrary vocabulary. Learning grammar is equivalent to language modeling, as the support of p will represent the set of all grammatically correct sentences. Furthermore, let j α represent a particular language. Let p jα (·) represent the language model for the j α th language and w jα represents a word from the j α th language. Let k jα represent a distributed representation of a specific language along the lines of the PnP argument . UG, through the lens of statistical language modeling, hypothesizes the existence of a factorization of p jα (·) containing a language agnostic segment. The factorization used throughout this paper is the following (• denotes function composition): DISPLAYFORM1 The distribution matching constraint d, insures that the representations across languages are common as hypothesized by the UG argument. as a distributed representation of the language of size f and returns a language specific decoded representation. e −1 jα maps our decoded representation back to the token space. For the purposes of distribution matching we utilize the GAN framework. Following recent successes we use Wasserstein-1 as our distance function d.Given two languages j α and j β the distribution of the universal representations should be within with respect to the W 1 of each other. Using the Kantarovich-Rubenstein duality BID37 we define DISPLAYFORM2 where L is the Lipschitz constant of f. Throughout this paper we satisfy the Lipschitz constraint by clamping the parameters to a compact space, as done in the original WGAN paper. Therefore the complete loss function for m languages each containing N documents becomes: DISPLAYFORM3 λ is a scaling factor for the distribution constraint loss. Our specific implementation of our factorization and optimization problem we denote as UG-WGAN. Each function described in the previous section we implement using neural networks. For e jα in equation 1 we use a language specific embedding table followed by a LSTM BID17 jα is non trivial therefore we use another language specific LSTM whose outputs we multiply by the transpose of the embedding table of e jα to obtain token probabilities. For regularization we utilized standard dropout after the embedding layers and layer-wise locked dropout after each LSTM's layer BID34 BID14.The critic, adopting the terminology from, takes the input from u, feeds it through a stacked LSTM, aggregates the hidden states using linear sequence attention as described in DrQA BID8. Once we have the aggregated state we map to a m × m matrix from where we can compute the total Wasserstein loss. A Batch Normalization layer is appended to the end of the critic BID19. The α, βth index in the matrix correspond to the function output of f in calculating DISPLAYFORM0 We trained UG-WGAN with a variety of languages depending on the downstream task. For each language we utilized the respective Wikipedia dump. From the wikipedia dump we extract all pages using the wiki2text 1 utility and build language specific vocabularies consisting of 16k BPE tokens BID33. During each batch we uniform sample random documents from our set of languages which are approximately the same length, therefore a batch will be mixed with respect to language. We train our language model via BPTT where the truncation length progressively grows from 15 to 50 throughout training. The critic is updated 10 times for every update of the language model. We trained each language model for 14 days on a NVidia Titan X. For each language model we would do a sweep over λ, but in general we have found that λ = 0.1 works sufficiently well for minimizing both perplexity and Wasserstein distance. A couple of interesting questions arise from the described training procedure. Is the distribution matching constraint necessary or will simple joint language model training exhibit the properties we're interested in? Can this optimization process fundamentally learn individual languages grammar while being constrained by a universal channel? What commonalities between languages can we learn and are they informative enough to be exploited?We can test out the usefulness of the distribution matching constraint by running an ablation study on the λ hyper-parameter. We trained UG-WGAN on English, Spanish and Arabic wikidumps following the procedure described above. We kept all the hyper-parameters consistent apart for augmenting λ from 0 to 10. The are shown in Figure 2. Without any weight on the distribution matching term the critic trivially learns to separate the various languages and no further training reduces the wasserstein distance. The joint language model internally learns individual language models who are partitioned in the latent space. We can see this by running a t-SNE plot on the universal (u(·)) representation of our model and seeing existence of clusters of the same language as we did in FIG4 BID25 ). An universal model satisfying the distribution matching constrain would mix all languages uniformly within it's latent space. To test the universality of UG-WGAN representations we will apply them to a set of orthogonal NLP tasks. We will leave the discussion on the learnability of grammar to the Discussion section of this paper. By introducing a universal channel in our language model we reduced a representations dependence on a single language. Therefore we can utilize an arbitrary set of languages in training an auxiliary task over UG encodings. For example we can train a downstream model only on one languages data and transfer the model trivially to any other language that UG-WGAN was trained on. To test the universality of UG-WGAN representation we first trained UG-WGAN in English, Chinese and German following the procedure described in Section 4. The embedding size of the table was 300 and the internal LSTM hidden size was 512. A dropout rate of 0.1 was used and trained with the ADAM optimization method BID21 ). Since we are interested in the zeroshot capabilities of our representation, we trained our sentiment analysis model only on the English IMDB Large Movie Review dataset and tested it on the Chinese ChnSentiCorp dataset and German SB-10K BID24 BID36. We binarize the label's for all the datasets. Our sentiment analysis model ran a bi-directional LSTM on top of fixed UG representations from where we took the last hidden state and computed a logistic regression. This was trained using standard SGD with momentum. IMDB ChnSentiCorp SB-10KNMT + Logistic BID32 12.44% 20.12% 22.92% FullUnlabeledBow BID24 11.11% * * NB-SVM TRIGRAM BID27 8.54% 18.20% 19.40% UG-WGAN λ = 0.1 + Logistic (Ours) 8.01% 15.40% 17.32% UG-WGAN λ = 0.0 + Logistic (Ours) 7.80% 53.00% 49.38% Sentiment Neuron BID31 7.70% * * SA-LSTM BID13 7.24% * * We also compare against encodings learned as a by-product of multi-encoder and decoder neural machine translation as a baseline BID22. We see that UG representations are useful in situations when there is a lack of data in an specific language. The language agnostics properties of UG embeddings allows us to do successful zero-shot learning without needing any parallel corpus, furthermore the ability to generalize from language modeling to sentiment attests for the universal properties of these representations. Although we aren't able to improve over the state of the art in a single language we are able to learn a model that does surprisingly well on a set of languages without multilingual data. A natural language inference task consists of two sentences; a premise and a hypothesis which are either contradictions, entailments or neutral. Learning a NLI task takes a certain nuanced understanding of language. Therefore it is of interest whether or not UG-WGAN captures the necessary linguistic features. For this task we use the Stanford NLI (sNLI) dataset as our training data in English BID7. To test the zero-shot learning capabilities we created a russian sNLI test set by random sampling 400 sNLI test samples and having a native russian speaker translate both premise and hypothesis to russian. The label was kept the same. For this experiment we trained UG-WGAN on the English and Russian language following the procedure described in Section 4. We kept the hyper-parameters equivalent to the Sentiment Analysis experiment. All of the NLI model tested were run over the fixed UG embeddings. We trained two different models from literature, Densely-Connected Recurrent and Co-Attentive Network by BID20 and Multiway Attention Network by BID35. Please refer to this papers for further implementation details. Densely-Connected Recurrent and Co-Attentive Network Ensemble BID20 9.90% * UG-WGAN (λ = 0.1) + Densely-Connected Recurrent and Co-Attentive Network BID20 12.25% 21.00%UG-WGAN (λ = 0.1) + Multiway Attention Network BID35 21.50% 34.25% UG-WGAN (λ = 0.0) + Multiway Attention Network BID35 13.50% 65.25% UG-WGAN (λ = 0.0) + Densely-Connected Recurrent and Co-Attentive Network BID20 11.50% 68.25%Unlexicalized features + Unigram + Bigram features BID7 21.80% 55.00% UG representations contain enough information to non-trivially generalize the NLI task to unseen languages. That being said, we do see a relatively large drop in performance moving across languages which hints that either our calculation of the Wasserstein distance may not be sufficiently accurate or the universal representations are biased toward specific languages or tasks. One hypothesis might be that as we increase λ the cross lingual generalization gap (difference in test error on a task across languages) will vanish. To test this hypothesis we conducted the same experiment where UG-WGAN was trained with a λ ranging from 0 to 10. From each of the experiments we picked the model epoch which showed the best perplexity. The NLI specific model was the Densely-Connected Recurrent and Co-Attentive Network. Increasing λ doesn't seem to have a significant impact on the generalization gap but has a large impact on test error. Our hypothesis is that a large λ doesn't provide the model with enough freedom to learn useful representations since the optimizations focus would largely be on minimizing the Wasserstein distance, while a small λ permits this freedom. One reason we might be seeing this generalization gap might be due to the way we satisfy the Lipschitz constraint. It's been shown that there are better constraints than clipping parameters to a compact space such as a gradient penalty BID15. This is a future direction that can be explored. Universal Grammar also comments on the learnability of grammar, stating that statistical information alone is not enough to learn grammar and some form of native language faculty must exist, sometimes titled the poverty of stimulus (POS) argument BID10 BID23 ). The goal of our paper is not to make a statement on the Universal Grammar hypothesis. But from a machine learning perspective, we're interested in extracting informative features. That being said it is of interest to what extent language models capture grammar and furthermore the extent to which models trained with our objective learn grammar. One way to measure universality is by studying perplexity of our multi-lingual language model as we increase the number of languages. To do so we trained 6 UG-WGAN models on the following languages: English, Russian, Arabic, Chinese, German, Spanish, French. We maintain the same procedure as described above. The hidden size of the language model was increased to 1024 with 16K BPE tokens being used. The first model was trained on English Russian, second was trained on English Russian Arabic and so on. For Arabic we still trained from left to right even though naturally the language is read from right to left. We report the in FIG3. As we increase the number of languages the perplexity gap between constrained and unconstrained UG-WGAN (λ = 0.0) decreases which implies while controlling capacity, our constrained (universal λ = 0.1) language model, models language (almost) as well as jointly trained language models with no universal constraints (λ = 0.0).Furthermore, the heatmap in FIG3 shows the perplexity gap of UG-WGAN trained on any combination of 2 languages from our set of 7. We can treat the perplexities as a loose measure of distance λ = 0.0 λ = 0.1 en earth's oxide is a monopoly that occurs towing of the carbon-booed trunks, ing in a beam containing of oxygen through the soil, salt, warm waters, and the different proteins.the practice of epimatic behaviours may be required in many ways of all non-traditional entities.the groove and the products are numeric because they are called "pressibility" (ms) nutrients containing specific different principles that are available from the root of their family, including a wide variety of molecular and biochemical elements. a state line is a self-government environment for statistical cooperation, which is affected by the monks of canada, the east midland of the united kingdom.however, compared to the listing of special definitions, it has evolved to be congruent with structural introductions, allowing to form the chemical form.the vernacular concept of physical law is not as an objection (the whis) but as a universal school.es la revista ms reciente vari el manuscrito originalmente por primera vez en la revista publicada en 1994.en el municipio real se localiza al mar del norte y su entorno en escajros alto, con mayor variedad de cclica poblacin en forma de cerca de 1070 km2.de hecho la primera cancin de "blebe cantas", pahka zanjiwtryinvined cot de entre clases de fanticas, apareci en el ornitlogo sello triusion, jr., en la famosa publicacin playboy de john allen.fue el ltimo habitantes de suecia, con tres hijos, atasaurus y aminkinano (nuestra).The names of large predators in charlesosaurus include bird turtles hibernated by aerial fighters and ignored fish.jaime en veracruz fue llamado papa del conde mayor de valdechio, hijo de diego de ziga. We see from Figure 2 that perplexity worsens proportional to λ. We explore the differences by sampling sentences from an unconstrained language model and λ = 0.1 language model trained towards English and Spanish in Table 3. In general there is a very small difference between a language model trained with our objective and one without. The constrained model tends to make more gender mistakes and mistakes due to Plural-Singular Form in Spanish. In English we saw virtually no fundamental differences between the language models. One explanation of this phenomena comes from the autonomy of syntax argument, which argues that semantics have no weight on syntax BID16. Our hypothesis is that both models learn syntax well, but the models with better perplexity generate sentences with better or clearer semantic meaning. Although completely learning grammar from statistical signals might be improbable, we can still extract useful information. In this paper we introduced an unsupervised approach toward learning language agnostic universal representations by taking inspiration from the Universal Grammar hypothesis. We showed that we can use these representations to learn tasks in one language and automatically transfer them to others with no additional training. Furthermore we studied the importance of the Wasserstein constraint through the λ hyper-parameter. And lastly we explored the difference between a standard multi-lingual language model and UG-WGAN by studying the generated outputs of the respective language models as well as the perplexity gap growth with respect to the number of languages.

Represent the short summary for finding the corresponding scientific article (long)

By taking inspiration from linguistics, specifically the Universal Grammar hypothesis, we learn language agnostic universal representations which we can utilize to do zero-shot learning across languages.

represent the tldr for finding the corresponding scientific article

Fewshot example: "Better adversarial training by learning to map back to the data manifold with autoencoders in the hidden states.  " == "Deep networks have achieved impressive across a variety of important tasks. However, a known weakness is a failure to perform well when evaluated on data which differ from the training distribution, even if these differences are very small, as is the case with adversarial examples. We propose \emph{Fortified Networks}, a simple transformation of existing networks, which “fortifies” the hidden layers in a deep network by identifying when the hidden states are off of the data manifold, and maps these hidden states back to parts of the data manifold where the network performs well. Our principal contribution is to show that fortifying these hidden states improves the robustness of deep networks and our experiments (i) demonstrate improved robustness to standard adversarial attacks in both black-box and white-box threat models; (ii) suggest that our improvements are not primarily due to the problem of deceptively good due to degraded quality in the gradient signal (the gradient masking problem) and (iii) show the advantage of doing this fortification in the hidden layers instead of the input space. We demonstrate improvements in adversarial robustness on three datasets (MNIST, Fashion MNIST, CIFAR10), across several attack parameters, both white-box and black-box settings, and the most widely studied attacks (FGSM, PGD, Carlini-Wagner). We show that these improvements are achieved across a wide variety of hyperparameters. The success of deep neural networks across a variety of tasks has also driven applications in domains where reliability and security are critical, including self-driving cars BID6, health care, face recognition BID25, and the detection of malware BID17. Security concerns arise when an agent using such a system could benefit from the system performing poorly. Reliability concerns come about when the distribution of input data seen during training can differ from the distribution on which the model is evaluated. Adversarial examples BID11 from attacks on neural network models, applying small perturbations to the inputs that change the predicted class. Such perturbations can be small enough to be unnoticeable to the naked eye. It has been shown that gradient-based methods allow one to find modifications of the input that often change the predicted class BID26 BID11. More recent work demonstrated that it is possible to create modifications such that even when captured through a camera, they change the predicted class with high probability BID7.Some of the most prominent classes of defenses against adversarial examples include feature squeezing BID29, adapted encoding of the input (Jacob BID14, and distillation-related approaches BID20 . Existing defenses provide some robustness but most are not easy to deploy. In addition, many have been shown to be providing the illusion of defense by lowering the quality of the gradient signal, without actually providing improved robustness BID1 . Still others require training a generative model directly in the visible space, which is still difficult today even on relatively simple datasets. Our work differs from the approaches using generative models in the input space in that we instead employ this robustification on the distribution of the learned hidden representations, which makes the The plot on the right shows direct experimental evidence for this hypothesis: we added fortified layers with different capacities to MLPs trained on MNIST, and display the value of the total reconstruction errors for adversarial examples divided by the total reconstruction errors for clean examples. A high value indicates success at detecting adversarial examples. Our support the central motivation for fortified networks: that off-manifold points can much more easily be detected in the hidden space (as seen by the relatively constant ratio for the autoencoder in hidden space) and are much harder to detect in the input space (as seen by this ratio rapidly falling to zero as the input-space autoencoder's capacity is reduced).identification of off-manifold examples easier. We do this by training denoising autoencoders on top of the hidden layers of the original network. We call this method Fortified Networks. We demonstrate that Fortified Networks (i) can be generically added into an existing network; (ii) robustify the network against adversarial attacks and (iii) provide a reliable signal of the existence of input data that do not lie on the manifold on which it the network trained. In the sections that follow, we discuss the intuition behind the fortification of hidden layers and lay out some of the method's salient properties. Furthermore, we evaluate our proposed approach on MNIST, Fashion-MNIST, CIFAR10 datasets against whitebox and blackbox attacks. The Empirical Risk Minimization Framework Let us consider a standard classification task with an underlying data distribution D over pairs of examples x ∈ R d and corresponding labels y ∈ [k]. We also assume that we are given a suitable loss function L(θ, x, y), for instance the cross-entropy loss. As usual, θ ∈ R p is the set of model parameters. Our goal then is to find model parameters θ that minimize the risk E (x,y)∼D [L(x, y, θ) ]. This expectation cannot be computed, therefore a common approach is to to minimize the empirical risk 1/N D L(x, y, θ) taking into account only the examples in a given dataset D.Adversarial Attacks and Robustness While the empirical risk minimization framework has been very successful and often leads to excellent generalization, it has the significant limitation that it doesn't guarantee robustness, and more specifically performance on examples off the data manifold. BID19 proposed an optimization view of adversarial robustness, in which the adversarial robustness of a model is defined as a min-max problem, DISPLAYFORM0 where S denotes the set of all points within a sphere of radius ε, which is task-specific. Larger ε values correspond to stronger attacks but which may be more visually apparent. Denoising Autoencoders Denoising autoencoders (DAEs) are neural networks which take a noisy version of an input (for example, an image) and are trained to predict the noiseless version of that input. This approach has been widely used for feature learning and generative modeling in deep learning BID4. More formally, denoising autoencoders are trained to minimize a reconstruction error or negative log-likelihood of generating the clean input. For example, with DISPLAYFORM1 Figure 2: Diagram illustrating a one-layer fortified network. A network is evaluated with a data sample x and its corresponding adversarial example x. Hidden units h k and h k are corrupted with noise, encoded with the encoder enc., and decoded with the decoder dec. The autoencoder (denoted by the red color) is trained to reconstruct the hidden unit h k that corresponds to the clean input. Dotted lines are two reconstruction costs: for a benign (L rec) and adversarial examples (L adv).Gaussian log-likelihood of the clean input given the corrupted input, r θ the learned denoising function, C a corruption function with Gaussian noise of variance σ 2, the reconstruction loss is BID0 demonstrated that with this loss function, an optimally trained denoising autoencoder's reconstruction vector is proportional to the gradient of the log-density: DISPLAYFORM2 DISPLAYFORM3 This theory establishes that the reconstruction vectors from a well-trained denoising autoencoder form a vector field which points in the direction of the data manifold. However, BID0 showed that this may not hold for points which are distant from the manifold, as these points are rarely sampled during training. In practice, denoising autoencoders are not just trained with tiny noise but also with large noises, which blurs the data distribution as seen by the learner but makes the network learn a useful vector field even far from the data. We propose the use of DAEs inserted at crucial points between layers of the original neural network in order to denoise the transformed data points which may lie away from the original data manifold. Intuitively, the method aims to regularize the hidden representations by keeping the activations on the surface of the corresponding projected data manifold through the application of a DAE trained on the hidden representations (on the original clean data). We argue that applying the DAEs on the hidden layers-as opposed to the raw input signal-facilitates learning, while providing a stronger protection from adversarial attacks. As illustrated in FIG0, we hypothesize that more abstract representations associated with deeper networks are easier to denoise because the transformed data manifolds are flatter. The flattening of data manifolds in the deeper layers of a neural network was first noted experimentally by BID5. We provide experimental support for these claims in Section 4.Layer fortification R Our method works by substituting a hidden layer h k with a denoised version. We feed the signal h k through the encoder network, E k, and decoder network, D k, of a DAE for layer k, which yields the denoised version, h DISPLAYFORM0 where n k is white Gaussian noise of variance σ 2 and appropriate shape. We call the ing layer, a fortified layer and the ing network the fortified network corresponding to the original network.• Reconstruction loss. For a mini-batch of N clean examples, DISPLAYFORM1 is fed into a DAE loss, similar to equation 2: DISPLAYFORM2 • Adversarial loss. We use some adversarial training method to produce the perturbed version of the mini-batch, DISPLAYFORM3, where x (i) is a small perturbation of x (i) which is designed to make the network produce the wrong answer. The corresponding hidden layer h DISPLAYFORM4 (using the perturbed rather than original input) is fed into a similar DAE loss: DISPLAYFORM5 where we note that the target reconstruction for denoising is the clean version of the hidden layer, without noise and without adversarial perturbation. For training purposes, we treat the DAEs as part of the fortified network, backpropagate through and train all weights jointly. Aside from the original classification loss, L c, we also include the classification loss from the adversarial objective, L c (y), and we introduce a dual objective for the DAEs. To build a fortified network, we can apply this fortification process to some or all the layers. The final objective used for training the fortified network includes the classification loss and all reconstruction and adversarial losses: DISPLAYFORM6 where λ rec > 0 and λ adv > 0 tune the strength of the DAE terms. This kind of training process allows for the production of hidden representations robust to small perturbations, and in particular, to adversarial attacks. Off-manifold signaling The reconstruction losses act as a reliable signal for detecting off-manifold examples (cf. Section 4). This is a particularly useful property in practice: not only can we provide more robust classification , we can also sense and suggest to the analyst or system when the original example is either adversarial or from a significantly different distribution. Motivation for when and where to use fortified layers We have discussed advantages to placing fortified layers in the hidden states instead of the input space (with further discussion in section 6,) but the question of where exactly fortified layers need to be placed remains unanswered. Is it just the final hidden layer? Is it every hidden layer? We outline two important considerations regarding this issue. First, in the higher-level hidden layers, it is much easier for the network to identify points which are off of the manifold or close to the margin. This is directly experimentally demonstrated in fig. 1. Secondly, the higher level hidden layers may already look like points that are not adversarial due to the effect of the adversarial perturbations in the earlier layers. While we are not aware of any formal study of this phenomenon, it is clearly possible (imagine for example a fortified layer on the output from the softmax, which could only identify unnatural combinations of class probabilities.) Given these opposing objectives, we argue for the inclusion of multiple fortified layers across the network. We evaluated the performance of our model as a defense against adversarial attacks. We focused on two of the most popular and well-studied attacks. Firstly, we consider the Fast Gradient Sign Method (FGSM, BID11 which is popular as it only requires a single step and can still be effective against many networks. Secondly, we consider the projected gradient descent attack BID16 which is slower than FGSM as it requires many iterations, but has been shown to be a much stronger attack BID19 .Additionally, we consider both white-box attacks (where the attackers knows the model) and blackbox attacks (where they don't, but they have access to the training set.)Fast Gradient Sign Method. The Fast Gradient Sign Method (FGSM) BID11 is a simple one-step attack that produces ∞ -bounded adversaries via the following gradient based perturbation. DISPLAYFORM0 Projected Gradient Descent. The projected gradient descent attack BID19, sometimes referred to as FGSM k, is a multi-step extension of the FGSM attack characterized as follows: DISPLAYFORM1 initialized with x 0 as the clean input x and with the corrupted input x as the last step in the sequence. Π refers to the projection operator, which in this context means projecting the adversarial example back onto the region within an ε radius of the original data point, after each step in the adversarial attack. Finally we considered the Carlini-Wagner L2 attack BID8 which consists of joint optimization of loss maximization and minimizing the distance of the adversarial example to the original example. A significant challenge with evaluating defenses against adversarial attacks is that many attacks rely upon a network's gradient. Methods which reduce the quality of this gradient, either by making it flatter or noisier can lead to methods which lower the effectiveness of gradient-based attacks, but which are not actually robust to adversarial examples BID2 BID23. This process, which has been referred to as gradient masking or gradient obfuscation, must be analyzed when studying the strength of an adversarial defense. One method for studying the extent to which an adversarial defense gives deceptively good as a of gradient masking relies on the observation that black-box attacks are a strict subset of white-box attacks, so white-box attacks should always be at least as strong as black-box attacks. If a method reports much better defense against white-box attacks, it suggests that the selected white-box attack is underpowered as a of gradient masking. Another test for gradient masking is to run an iterative search, such as projected gradient descent (PGD) with an unlimited range for a large number of iterations. If such an attack is not completely successful, it indicates that the model's gradients are not an effective method for searching for adversarial images, and that gradient masking is occurring. Still another test is to confirm that iterative attacks with small step sizes always outperform single-step attacks with larger step sizes (such as FGSM). If this is not the case, it may suggest that the iterative attack becomes stuck in regions where optimization using gradients is poor due to gradient masking. Additionally, BID1 discussed the Backward Pass Differentiable Approximation (BPDA) attack to cover cases where a defense employs a transformation which is clearly nondifferentiable or reduces the quality of the gradients. Because we pass gradients through the fortified layers in the normal training of our network, it is unlikely that the quality of these gradients is significantly deteriorated, and there isn't a reason to expect that they would be because the fortified layers are relatively shallow and use normal activation functions (i.e. no non-differentiable functions.). Additionally we ran additional experiments using the identity function version of BPDA on Fashion MNIST FGSM (ε = 0.3) with standard deviation across five trials. Our adversarial training baseline achieved an accuracy of 84.64±0.48 on this task, fortified networks with the normal attack achieved an accuracy of 89.50±0.45, and fortified networks with the BPDA version of the attack (treating the autoencoder as an identity) achieved an accuracy of 89.88±0.30, which corresponds to a weaker attack. For details about the specifics of our model architectures and hyperparameters we refer readers to the Appendix. With all experiments, we use the same attacks (with identical parameters) at training and test time to generate adversarial examples. An important point to note here is that all of the autoencoders in our fortified layers used a single hidden layer with tied weights. We also performed an analytical experiment on an RNN language model to study if fortified networks can detect when it's being given outputs from its own model (sampling mode) when trained using ground truth input sequences. To this end we train a language model on the standard Text8 dataset, which is derived from Wikipedia articles. We trained a single-layer LSTM with 1000 units at the character-level, and included fortified layers between the hidden states and the output on each time step. With 50 sampling steps, the fortified layers had a reconstruction error on average 103% of the teacher forcing reconstruction error. With 180 sampling steps, this valued increased to 112%. With 300 sampling steps this increased even further to 134%. This is clear evidence that the outputs move off of the manifold with more sampling steps, and that this is effectively measured by fortified networks. We ran with many hyperparameters for the fortified layers to demonstrate the generality of the improvement given by fortified networks. We ran FGSM on Fashion-MNIST (ε=0.3) while varying the amount of noise injected and the weighting on the reconstruction loss. We also varied the amount of noise and choice of losses in the ablation experiment in Table 7. We achieved consistent improvement across a variety of settings but found the largest improvement when using both the adversarial and clean reconstruction losses and a small amount of noise. Thus, we see that a consistent improvement is achieved when the weighting on the reconstruction part of the loss and the amount of noise injected are varied over several orders of magnitude. Using Generative Models as a Defense. The observation that adversarial examples often consist of points off of the data manifold and that deep networks may not generalize well to these points motivated BID12 BID24 BID18 to consider the use of the generative models as a defense against adversarial attacks. Ilyas et al. FORMULA0; BID10 also showed the existence of adversarial examples which lie on the data manifold, and showed that training against adversarial examples forced to lie on the manifold is an effective defense. Our method shares a closely related motivation to these prior works, with a key difference being that we propose to consider the manifold in the space of learned representations, Table 2: CIFAR-10 PGD Results with (non-resnet) CNNs. In these experiment we used a fortified block (single convolutional autoencoder) following each convolutional layer. Both experiments were run for 200 epochs and with all hyperparameters and architecture kept the same with the exception of the fortified layer being added. We considered different types of baselines:'Baseline -no new layers' means we simply removed the fortified block.' Baseline -extra layers' means that we added extra layers to match the capacity of the fortified layers, but only gave half of these extra layers activations as the fortified block has two layers but only one activation.' Baseline -extra activations' means that we added an activation following each layer, giving more activations in total than the Fortified Network. Attack instead of considering the manifold directly in the visible space. One motivation for this is that the learned representations have a simpler statistical structure, which makes the task of modeling this manifold and detecting unnatural points much simpler. Learning the distribution directly in the visible space is still very difficult (even state of the art models fall short of real data on metrics like Inception Score) and requires a high capacity model. Additionally, working in the space of learned representations allows for the use of a relatively simple generative model, in our case a small denoising autoencoder. proposed to work around these challenges from working in the visible space by using the Deep Image Prior instead of an actual generative model. While this has the advantage of Table 4: Accuracies against white-box attacks on Fashion MNIST. For PGD we used ε = 0.1 and for FGSM we used ε = 0.1 and ε = 0.3 1. Compared with DefenseGAN BID24. Table 5: Left: Accuracies against blackbox MNIST attacks with adversarial training (FGSM). DISPLAYFORM0 Reporting 50/50 compared to previous works (, PS). The test error on clean examples is in parenthesis. Right:We ran a fortified network on Fashion-MNIST using adversarial training with PGD for a variety of ε values, each for 5 epochs. The motivation behind this experiment, suggested by BID1 is confirming if unbounded (ε = 1) adversarial attacks are able to succeed. A defense which succeeds primarily by masking or obfuscating the gradients would fail to bring the accuracy to zero even with an unbounded attack. As can be seen, unbounded attacks against Fortified Networks succeed when given a sufficiently large ε, which is evidence against gradient masking. being a model that doesn't require a special training procedure (as deep image prior is a separate optimization process for each example) it may be limited in the types of adversarial attacks that it's resistant to, and it would provide no defense against adversarial attacks which are in the range of a convolutional network, which have been shown to exist BID28.Another key difference between our work and BID24 is that both DefenseGAN and the Invert-and-Classify approach use an iterative search procedure at inference time to map observed data points onto nearby points on the range of the generator. On the other hand, our approach uses small denoising autoencoders that are used in the same way (i.e. a simple forward application) during both training and testing. The use of such an iterative procedure presents challenges for evaluation, as it is possible for gradients to vanish while doing backpropagation through such a procedure, which may lead to an overestimate in the strength of the defense due to the gradient masking problem BID22 BID1. One indicator of the gradient masking problem is black-box attacks outperforming white-box attacks, which is an indicator of under-powered attacks as black-box attacks are a strict subset of white-box attacks. This indicator of gradient obfuscation was present in the work of BID24 where black-box attacks were generally stronger against their defense, but with our method we observe very similar defense quality against black-box and white-box attacks. BID12 BID18 both considered using an autoencoder as a pre-processing step in the input space. Interestingly BID18 ) used a loss function defined in the space of the hidden states, but still used autoencoders directly in the input space. Adversarial Hidden State Matching. BID9 demonstrate that adversarially matching the hidden layer activations of regular and adversarial examples improves robustness. This work shared the same motivation of using the hidden states to improve robustness, but differed in that they used an adversarial objective and worked in the original hidden states instead of using a generative model (in our case, the DAE in the fortified layers). We present direct experimental comparisons with their work in section 5. BID15 proposed a method which involves matching the logit (pre-softmax outputs) values for the original samples with the logit values ing from adversarial examples. Denoising Feature Matching. BID27 proposed to train a denoising autoencoder in the hidden states of the discriminator in a generative adversarial network. The generator's parameters are then trained to make the reconstruction error of this autoencoder small. This has the effect of encouraging the generator to produce points which are easy for the model to reconstruct, which will include true data points. Both this and Fortified Networks use a learned denoising autoencoder in the hidden states of a network. A major difference is that the denoising feature matching work focused on generative adversarial networks and tried to minimize reconstruction error through a learned generator network, whereas our approach targets the adversarial examples problem. Additionally, our objective encourages the output of the DAE to denoise adversarial examples so as to point back to the hidden state of the original example, which is different from the objective in the denoising feature matching work, which encouraged reconstruction error to be low on states from samples from the generator network. Adversarial Spheres. BID10 studied the existence of adversarial examples in the task of classifying between two hollow concentric shells. Intriguingly, they prove and construct adversarial examples which lie on the data manifold (although also looked for such examples experimentally using GANs.) The existence of such on-manifold adversarial examples demonstrates that a simplified version of our model trained with only L rec could not protect against all adversarial examples. However, training with L adv encourages the fortified layers to map back from points which are not only off of the manifold, but also to map back from points which are hard to classify, allowing Fortified Networks to also potentially help with on-manifold adversarial examples as well. Protecting against adversarial examples could be of paramount importance in mission-critical applications. We have presented Fortified Networks, a simple method for the robustification of existing deep neural networks. Our method is practical, as fortifying an existing network entails introducing DAEs between the hidden layers of the network, which can be automated. Furthermore, the DAE reconstruction error at test time is a reliable signal of distribution shift, which can in examples unlike those encountered during training. High error can signify either adversarial attacks or significant domain shift; both are important cases for the analyst or system to be aware of. Moreover, fortified networks are efficient: since not every layer needs to be fortified to achieve improvements, fortified networks are an efficient way to improve robustness to adversarial examples. For example, we have shown improvements on ResNets where only two fortified layers are added, and thus the change to the computational cost is very slight. Finally, fortified networks are effective, as they improve on adversarial defense on three datasets (MNIST, Fashion MNIST, and CIFAR10), across a variety of attack parameters (including the most widely used ε values), across three widely studied attacks (FGSM, PGD, Carlini-Wagner L2), and in both the black-box and white-box settings. A EXPERIMENTAL SETUP All attacks used in this work were carried out using the Cleverhans BID21 ) library. A.1 WHITE-BOX ATTACKS Our convolutional models (Conv, in the tables) have 2 strided convolutional layers with 64 and 128 filters followed by an unstrided conv layer with 128 filters. We use ReLU activations between layers then followed by a single fully connected layer. The convolutional and fully-connected DAEs have a single bottleneck layer with leaky ReLU activations with some ablations presented in the table below. With white-box PGD attacks, we used only convolutional DAEs at the first and last conv layers with Gaussian noise of σ = 0.01 whereas with FGSM attacks we used a DAE only at the last fully connected layer. The weight on the reconstruction error λ rec and adversarial cost λ adv were set to 0.01 in all white-box attack experiments. We used the Adam optimizer with a learning rate of 0.001 to train all models. The table below lists a few ablations with different activation functions in the autoencoder Our black-box are based on a fully-connected substitute model (input-200-200-output), which was subsequently used to attack a fortified convolutional network. The CNN was trained for 50 epochs using adversarial training, and the predictions of the trained CNN were used to train the substitute model. 6 iterations of Jacobian data augmentation were run during training of the substitute, with λ = 0.1. The test set data holdout for the adversary was fixed to 150 examples. The learning rate was set to 0.003 and the Adam optimizer was used to train both models. TAB0: More attack steps to uncover gradient masking effects. Different epsilon values at attack time. We applied attacks of different ε values to a network trained with ε = 0.03. Results shown in tab. 9.More attack iterations. We ran attacks for more steps at test time, on a network trained on attacks of 7 steps. Results shown in tab. 10."

We use a hypernetwork to predict optimal weights given hyperparameters, and jointly train everything together.