Question: Represent the article for finding a single-sentence TLDR

Question: Hierarchical planning, in particular, Hierarchical Task Networks, was proposed as a method to describe plans by decomposition of tasks to sub-tasks until primitive tasks, actions, are obtained. Plan verification assumes a complete plan as input, and the objective is finding a task that decomposes to this plan. In plan recognition, a prefix of the plan is given and the objective is finding a task that decomposes to the (shortest) plan with the given prefix. This paper describes how to verify and recognize plans using a common method known from formal grammars, by parsing. Hierarchical planning is a practically important approach to automated planning based on encoding abstract plans as hierarchical task networks (HTNs) BID3. The network describes how compound tasks are decomposed, via decomposition methods, to sub-tasks and eventually to actions forming a plan. The decomposition methods may specify additional constraints among the subtasks such as partial ordering and causal links. There exist only two systems for verifying if a given plan complies with the HTN model (a given sequence of actions can be obtained by decomposing some task). One system is based on transforming the verification problem to SAT BID2 and the other system is using parsing of attribute grammars BID1. Only the parsing-based system supports HTN fully (the SAT-based system does not support the decomposition constraints).Parsing became popular in solving the plan recognition problem BID5 as researchers realized soon the similarity between hierarchical plans and formal grammars, specifically context-free grammars with parsing trees close to decomposition trees of HTNs. The plan recognition problem can be formulated as the problem of adding a sequence of actions after some observed partial plan such that the joint sequence of actions forms a complete plan generated from some task (more general formulations also exist). Hence plan recognition can be seen as a generalization of plan verification. There exist numerous approaches to plan recognition using parsing or string rewriting (Avrahami-Zilberbrand and Kaminka 2005; BID5 BID4 BID5), but they use hierarchical models that are weaker than HTNs. The languages defined by HTN planning problems (with partial-order, preconditions and effects) lie somewhere between context-free (CF) and context-sensitive (CS) languages BID5 so to model HTNs one needs to go beyond the CF grammars. Currently, the only grammar-based model covering HTNs fully uses attribute grammars BID0. Moreover, the expressivity of HTNs makes the plan recognition problem undecidable BID2. Currently, there exists only one approach for HTN plan recognition. This approach relies on translating the plan recognition problem to a planning problem BID5, which is a method invented in BID5.In this paper we focus on verification and recognition of HTN plans using parsing. The uniqueness of the proposed methods is that they cover full HTNs including task interleaving, partial order of sub-tasks, and other decomposition constraints (prevailing constraints, specifically). The methods are derived from the plan verification technique proposed in BID1.There are two novel contributions of this paper. First, we will simplify the above mentioned verification technique by exploiting information about actions and states to improve practical efficiency of plan verification. Second, we will extend that technique to solve the plan (task) recognition problem. For plan verification, only the method in BID1 supports HTN fully. We will show that the verification algorithm can be much simpler and, hence, it is expected to be more efficient. For plan recognition, the method proposed in BID5 can in principle support HTN fully, if a full HTN planner is used (which is not the case yet as prevailing conditions are not supported). However, like other plan recognition techniques it requires the top task (the goal) and the initial state to be specified as input. A practical difference of out methods is that they do not require information about possible top (root) tasks and an initial state as their input. This is particularly interesting for plan/task recognition, where existing methods require a set of candidate tasks (goals) to select from (in principle, they may use all tasks as candidates, but this makes them inefficient). In this paper we work with classical STRIPS planning that deals with sequences of actions transferring the world from a given initial state to a state satisfying certain goal condition. World states are modelled as sets of propositions that are true in those states and actions are changing validity of certain propositions. Formally, let P be a set of all propositions modelling properties of world states. Then a state S ⊆ P is a set of propositions that are true in that state (every other proposition is false). Later, we will use the notation S + = S to describe explicitly the valid propositions in the state S and S − = P \ S to describe explicitly the propositions not valid in the state S.Each action a is described by three sets of propositions DISPLAYFORM0 a describes positive preconditions of action a, that is, propositions that must be true right before the action a. Some modeling approaches allow also negative preconditions, but these preconditions can be compiled away. For simplicity reasons we assume positive preconditions only (the techniques presented in this paper can also be extended to cover negative preconditions directly). Action a is applicable to state S iff B + a ⊆ S. Sets A + a and A − a describe positive and negative effects of action a, that is, propositions that will become true and false in the state right after executing the action a. If an action a is applicable to state S then the state right after the action a is: DISPLAYFORM1 γ(S, a) is undefined if an action a is not applicable to state S. The classical planning problem, also called a STRIPS problem, consists of a set of actions A, a set of propositions S 0 called an initial state, and a set of goal propositions G + describing the propositions required to be true in the goal state (again, negative goal is not assumed as it can be compiled away). A solution to the planning problem is a sequence of actions a 1, a 2,..., a n such that S = γ(...γ(γ(S 0, a 1), a 2 ),..., a n ) and G + ⊆ S. This sequence of actions is called a plan. The plan verification problem is formulated as follows: given a sequence of actions a 1, a 2,..., a n, and goal propositions G +, is there an initial state S 0 such that the sequence of actions forms a valid plan leading from S 0 to a goal state? In some formulations, the initial state might also be given as an input to the verification problem. To simplify the planning process, several extensions of the basic STRIPS model were proposed to include some control knowledge. Hierarchical Task Networks BID3 were proposed as a planning domain modeling framework that includes control knowledge in the form of recipes how to solve specific tasks. The recipe is represented as a task network, which is a set of sub-tasks to solve a given task together with the set of constraints between the sub-tasks. Let T be a compound task and ({T 1, ..., T k}, C) be a task network, where C are its constraints (see later). We can describe the decomposition method as a derivation (rewriting) rule: DISPLAYFORM0 The planning problem in HTN is specified by an initial state (the set of propositions that hold at the beginning) and by an initial task representing the goal. The compound tasks need to be decomposed via decomposition methods until a set of primitive tasks -actions -is obtained. Moreover, these actions need to be linearly ordered to satisfy all the constraints obtained during decompositions and the obtained plan -a linear sequence of actions -must be applicable to the initial state in the same sense as in classical planning. We denote action as a i, where the index i means the order number of action in the plan (a i is the i-th action in the plan). The state right after the action a i is denoted S i, S 0 is the initial state. We denote the set of actions to which a task T decomposes as act(T). If U is a set of tasks, we define act(U) = ∪ T ∈U act(T). The index of the first action in the decomposition of T is denoted start(T), that is, start(T) = min{i|a i ∈ act(T)}. Similarly, end(T) means the index of the last action in the decomposition of T, that is, end(T) = max{i|a i ∈ act(T)}.We can now define formally the constraints C used in the decomposition methods. The constraints can be of the following three types:• t 1 ≺ t 2: a precedence constraint meaning that in every plan the last action obtained from task t 1 is before the first action obtained from task t 2, end(t 1) < start(t 2), • before(U, p): a precondition constraint meaning that in every plan the proposition p holds in the state right before the first action obtained from tasks U, p ∈ S start(U)−1,• between(U, V, p): a prevailing condition meaning that in every plan the proposition p holds in all the states between the last action obtained from tasks U and the first action obtained from tasks V, DISPLAYFORM1 The HTN plan verification problem is formulated as follows: given a sequence of actions a 1, a 2,..., a n, is there an initial state S 0 such that the sequence of actions is a valid plan applicable to S 0 and obtained from some compound task? Again, the initial state might also be given as an input in some formulations. The HTN plan recognition problem is formulated as follows: given a sequence of actions a 1, a 2,..., a n, are there an initial state S 0 and actions a n+1,..., a n+m for some m ≥ 0 such that the sequence of actions a 1, a 2,..., a n+m is a valid plan applicable to S 0 and obtained from some compound task? In other words, the given actions form a prefix of some plan obtained from some compound task T. We will be looking for such a task T minimizing the value m (the number of added actions to complete the plan). If only the task T is of interest (not the actions a n+1, . . ., a n+m) then we are talking about the task (goal) recognition problem. The existing parsing-based HTN verification algorithm (Barták, Maillard, and Cardoso 2018) uses a complex structure of a timeline, that maintains the decomposition constraints so they can be checked when composing sub-tasks to a compound task. We propose a simplified verification method, that does not require this complex structure, as it checks all the constraints directly in the input plan. This makes the algorithm easier for implementation and presumably also faster. The novel hierarchical plan verification algorithm is shown in Algorithm 1. It first calculates all intermediate states (lines 2-8) by propagating information about propositions in action preconditions and effects. At this stage, we actually solve the classical plan validation problem as the algorithm verifies that the given plan is causally consistent (action precondition is provided by previous actions or by the initial state). The original verification algorithm did this calculation repeatedly each time it composed a compound task. It is easy to show that every action is applicable, that is, B + ai ⊆ S i−1 (lines 2 and 4). Next, we will show that DISPLAYFORM0. Right-to-left propagation (line 6) ensures that preconditions are propagated to earlier states if not provided by the action at a given position. In other words, if there is a proposition p ∈ S i+1 \ A + ai+1 then this proposition should be at S i. Line 6 adds such propositions to S i so it holds DISPLAYFORM1 then p would be deleted by the action a i+1, which means that the plan is not valid. The algorithm detects such situations (line 8).When the states are calculated, we apply a parsing algorithm to compose tasks. Parsing starts with the set of primitive tasks (line 9), each corresponding to an action from the input plan. For each task T, we keep a data structure describing the set act(T), that is, the set of actions to which the task decomposes. We use a Boolean vector I of the same size as the plan to describe this set; a i ∈ act(T) ⇔ I(i) = 1. To simplify checks of decomposition constraints, we also keep information about the index of first and last actions from act(T). Together, the task is represented using a quadruplet (T, s, e, I) in which T is a task, s is the index in the plan of the first action generated by T, e is the index in the plan of the last action generated by T (we say that [i, j] represents the interval of actions over which T spans), and I is a Boolean vector as described above. The algorithm applies each decomposition rule to compose a new task from already known sub-tasks (line 12). The composition consists of merging the sub-tasks, when we check that every action in the decomposition is obtained from a single sub-task (line 20), that is, act(T 0) = k j=1 act(T j) and ∀i = j: act(T i) ∩ act(T j) = ∅. We also check all the decomposition constraints; the code is direct Data: a plan P = (a 1, ..., a n) and a set of decomp. methods Result: a Boolean equal to true if the plan can be derived from some compound task, false DISPLAYFORM2 A i is a primitive task corresponding to action a i, I i is a Boolean vector of size n, such that ∀i ∈ 1..n, DISPLAYFORM3 foreach decomposition method R of the form If all tests pass, the new task is added to a set of tasks (line 25). Then we know that the task decomposes to actions, which form a sub-sequence (non necessarily continuous) of the plan to be verified. The process is repeated until a task that decomposes to all actions is obtained (line 27) or no new task can be composed (line 10). The algorithm is sound as the returned task decomposes to all actions in the input plan. If the algorithm finishes with the value false then no other task can be derived. As there is a finite number of possible tasks, the algorithm has to finish, so it is complete. DISPLAYFORM4 Any plan verification algorithm, for example, the one from the previous section, can be extended to plan recognition by feeding the verification algorithm with actions a 1,..., a n+k, where we progressively increase k. The actions a 1,..., a n are given as an input, while the actions a n+1,..., a n+k need to be generated (planned). However, this generate-and-verify approach would be inefficient for larger k as it requires exploration of all valid sequences of actions with the prefix a 1,..., a n. Assume that there could be 5 actions at the position n + 1 and 6 actions at the position n + 2. Then the generate-and-verify approach needs to explore up to 30 plans (not every action at the position n + 2 could follow every action at the position n + 1) and for each plan the verification part starts from scratch as the plans are different. This is where the verification algorithm from BID1 can be used as it does not require exactly one action at each position. The algorithm stores actions (sub-tasks) independently and only when it combines them to form a new task, it generates the states between the actions and checks the constraints for them. This resembles the idea of the Graphplan algorithm (Blum and Furst 1997). There are also sets of candidate actions for each position in the plan and the planextraction stage of the algorithm selects some of them to form a causally valid plan. We use compound tasks together with their decomposition constraints to select and combine the actions (we do not use parallel actions in the plan). The algorithm from (Barták, Maillard, and Cardoso 2018) extended to solve the plan recognition problem is shown in Algorithm 2. It starts with actions a 1,..., a n (line 2) and it finds all compound tasks that decompose to subsets of these actions (lines 4-30). This inner while-loop is taken from (Barták, Maillard, and Cardoso 2018), we only syntactically modified it to highlight the similarity with the verification algorithm from the previous section. If a task that decomposes to all current actions is found (line 30) then we are done. This is the goal task that we looked for and its timeline describes the recognized plan. Otherwise, we add all primitive tasks corresponding to possible actions at position n + 1 (line 33). Note that these are not parallel actions, the algorithm needs to select exactly one of them for the plan. Now, the parsing algorithm continues as it may compose new tasks that include one of those just added primitive tasks. Notice that the algorithm uses all composed tasks from previous iterations in succeeding iterations so it does not start from scratch when new actions are added. This process is repeated until the goal task is found. The algorithm is clearly sound as the task found is the task that decomposes to the shortest plan with a given prefix. This goes from the soundness and completeness of the verification algorithm (in particular, no task that decomposes to a shorter plan exists). The algorithm is semi-complete as if there exists a plan with the length n + k and with a given prefix, the algorithm will eventually find it at the (k + 1)-th iteration. If no plan with a given prefix exists then the algorithm will not stop. However, recall that the plan recognition problem is undecidable BID2 so any plan recognition approach suffers from this deficiency. Data: a plan P = (a 1, ..., a n), A i is a primitive task corresponding to action a i, and a set of decomposition methods Result: a Task that decomposes to a plan with prefix DISPLAYFORM0 DISPLAYFORM1 action a can be at position l; A is a primitive task for a} 34 goto 4Algorithm 2: Plan recognitionThe algorithm maintains a timeline for each compound task to verify all the constraints. This is the major difference from the above verification algorithm that points to the original plan. This timeline has been introduced in BID1, where all technical details can be found. We include a short description to make the paper selfcontained. A timeline is an ordered sequence of slots, where each slot describes an action, its effects, and the state right before the action. For task T, the actions in slots are exactly the actions from act(T). Both effects and states are modelled using two sets of propositions, Post + and Post − modeling positive and negative effects of the action and Pre + and Pre − modeling propositions that must and must not be the true in the state right before the action. Two sets are used as the state is specified only partially and propositions are added to it during propagation so it is necessary to keep information about propositions that must not be true in the state. The timeline always spans from the first to the last action of the task. Due to interleaving of tasks (actions from one task might be located between the actions of another task in the plan), some slots of the task might be empty. These empty slots describe "space" for actions of other tasks. When we are merging sub-tasks (lines 12-22), we merge their timelines, slot by slot. This is how the actions from sub-tasks are put together in a compound task. Notice, specifically, that it is not allowed for two merged sub-tasks to have actions in the same slot (line 15). This ensures that each action is generated by exactly one task. Data: a set of slots, a set of bef ore constraints Result: an updated set of slots 1 Function APPLYPRE(slots, pre) Propositions from bef ore and between constraints are "stored" in the corresponding slots (Algorithms 3 and 4) and their consistency is checked each time the slots are modified (line 26 of Algorithm 2). Consistency means that no proposition is true and false at the same state. Information between subsequent slots is propagated similarly to the verification algorithm (see Algorithm 5). Positive and negative propositions are now propagated separately taking in account empty slots. If there is no action in the slot then effects are unknown and hence propositions cannot be propagated. DISPLAYFORM2 DISPLAYFORM3 DISPLAYFORM4 Algorithm 5: Propagate A unique property of the proposed techniques is handling task interleaving -actions generated from different tasks may interleave to form a plan. This is the property that parsing techniques based on CF grammars cannot handle. Example in FIG4 demonstrates how the timelines are filled by actions as the tasks are being derived/composed from the plan. Assume, first, that a complete plan consisting of actions a 1, a 2,..., a 7 is given. The plan recognition algorithm can also handle such situations, when a complete plan is given, so it can serve for plan verification too (the verification variant of the Algorithm 2 should stop with a failure at line 33 as no action can be added during plan verification). In the first iteration, the algorithm will compose tasks T 2, T 3, T 4 as these tasks decompose to actions directly. Notice, how the timelines with empty slots are constructed. We know where the empty slots are located as we know the exact location of actions in the plan. In the second iteration, only the task T 1 is composed from already known tasks T 3 and T 4. Notice how the slots from these tasks are copied to the slots of a new timeline for T 1. By contrary, the slots in original tasks remain untouched as these tasks may merge with other tasks to form alternative decomposition trees (see the discussion below). Finally, in the third iteration, tasks T 1 and T 2 are merged to a new task T 0 and the algorithm stops there as a complete timeline that spans the plan fully is obtained (condition at line 30 of Algorithm 2 is satisfied).Let us assume that there is a constraint between({a 1}, {a 3}, p) in the decomposition method for T 3. This constraint may model a causal link between a 1 and a 3. When composing the task T 3, the second slot of its timeline remains empty, but the proposition p is placed there (see Algorithm 4). This proposition is then copied to the timeline of task T 1, when merging the timelines (line 17 of Algorithm 2), and finally also to the timeline of task T 0. During each merge operation, the algorithm checks that p can still be in the slot, in particular, that p is not required to be false at the same slot (line 26 repeatedly checks the constraints from methods. The new plan verification algorithm (Algorithm 1) handles the method constraints more efficiently as it uses the complete plan with states to check them. Moreover, the propagation of states is run just once in Algorithm 1 (lines 2-8), while Algorithm 2 runs it repeatedly each time the task is composed from subtasks. Hence, each constraint is verified just once in Algorithm 1, when a new task is composed. In particular, the constraint between({a 1}, {a 3}, p) is verified with respect to the states when task T 3 is introduced. Otherwise, both Algorithm 1 and Algorithm 2 derive the tasks in the same order (if the decomposition methods are explored in the same order). Instead of timelines, Algorithm 1 uses the Boolean vector I to identify actions belonging to each task. For example, for task T 3 the vector is and for task T 4 it is. When composing task T 1 from T 3 and T 4 the vectors are merged to get (see the loop at line 17). Notice that the vector always spans the whole plan, while the timelines start at the first action and finish with the last action of the task (and hence the same timeline can be used for different plan lengths).Assume now that only plan prefix consisting of a 1, a 2,..., a 6 is given. The plan recognition algorithm (Algorithm 2) will first derive tasks T 3 and T 4 only. Specifically, task T 2 cannot be derived yet as action a 7 is not in the plan. In the second iteration, the algorithm will derive task T 1 by merging tasks T 3 and T 4, exactly as we described above. As no more tasks can be derived, the inner loop finishes and the algorithm attempts to add actions that can follow the prefix a 1, a 2,..., a 6 (line 33). Let action a 7 be added at the 7-th position in the plan; actually all actions, that can follow the prefix, will be added as separate primitive tasks at position 7. Now the inner loop is restarted and task T 2 will be added in its first iteration. In the next iteration, task T 0 will be added and this will be the final task as it satisfies the condition at line 30.Assume, hypothetically, that the verification Algorithm 1 is used there. When it is applied to plan a 1, a 2,..., a 6, the algorithm derives tasks T 1, T 3, T 4 and fails as no task spans the whole plan and no more tasks can be derived. After adding action a 7, the algorithm will start from scratch as the states might be different due to propagating some propositions from the precondition of a 7. Hence, the algorithm needs to derive the tasks T 1, T 3, T 4 again and it will also add tasks T 0, T 2 and then it will finish with success. It may happen, that action a 5 can also be consistently placed to position 7. Then, we can derive two versions of task Let us denote the second version as T 1. The Algorithm 1 will stop there as no more tasks can be derived. Notice that tasks T 1, T 3, T 4 were derived repeatedly. If we try a 5 earlier than a 7 at position 7 then tasks T 1, T 3, T 4 will actually be generated three times before the algorithm finds a complete plan. Contrary, Algorithm 2 will add actions a 5 and a 7 together as two possible primitive tasks at position 7. It will use tasks T 1, T 3, T 4 from the previous iteration, it will add tasks T 1, T 3 as they can be composed from the primitive tasks (using the last a 5), it will also add tasks T 0, T 2 (using the last a 7), and will finish with success. Notice that T 1 cannot be merged with T 2 to get a new T 0 as T 1 has action a 5 at the 7-th slot while T 2 has a 7 there so the timelines cannot be merged (line 15 of Algorithm 2). In the paper, we proposed two versions of the parsing technique for verification of HTN plans and for recognition of HTN plans. As far as we know, these are the only approaches that currently cover HTN fully including all decomposition constraints. Both versions can be applied to solve both verification and recognition problems, but as we demonstrated using an example, each of them has some deficiencies when applied to the other problem. The next obvious step is implementation and empirical evaluation of both techniques. There is no doubt that the novel verification algorithm is faster than the previous approaches BID2 and BID1. The open question is how much faster it will be, in particular for large plans. The efficiency of the novel plan recognition technique in comparison to existing compilation technique BID5 ) is less clear as both techniques use different approaches, bottom-up vs. top-down. The disadvantage of the compilation technique is that it needs to re-generate the known plan prefix, but it can exploit heuristics to remove some overhead there. Contrary, the parsing techniques looks more like generate-and-test, but controlled by the hierarchical structure. It also guarantees finding the shortest extension of plan prefix.

represent the tldr for finding the corresponding scientific article

The paper describes methods to verify and recognize HTN plans by parsing of attribute grammars.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

This paper introduces partial grounding to tackle the problem that arises when the full grounding process, i.e., the translation of a PDDL input task into a ground representation like STRIPS, is infeasible due to memory or time constraints.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Neural architecture search (NAS), the task of finding neural architectures automatically, has recently emerged as a promising approach for unveiling better models over human-designed ones. However, most success stories are for vision tasks and have been quite limited for text, except for a small language modeling setup. In this paper, we explore NAS for text sequences at scale, by first focusing on the task of language translation and later extending to reading comprehension. From a standard sequence-to-sequence models for translation, we conduct extensive searches over the recurrent cells and attention similarity functions across two translation tasks, IWSLT English-Vietnamese and WMT German-English. We report challenges in performing cell searches as well as demonstrate initial success on attention searches with translation improvements over strong baselines. In addition, we show that on attention searches are transferable to reading comprehension on the SQuAD dataset. There has been vast literature on finding neural architectures automatically dated back to the 1980s with genetic algorithms BID18 to recent approaches that use random weights BID17, Bayesian optimization BID23, reinforcement learning BID1 BID28, evolution BID16, and hyper networks BID3. Among these, the approach of neural architecture search (NAS) using reinforcement learning by, barring computational cost, has been most promising, yielding stateof-the-art performances on several popular vision benchmarks such as CIFAR-10 and ImageNet. Building on NAS, others have found better optimizers BID2 and activation functions BID15 than human-designed ones. Despite these success stories, most of the work mainly focuses on vision tasks, with little attention to language ones, except for a small language modeling task on the Penn Tree Bank dataset (PTB) in.This work aims to bridge that gap by exploring neural architecture search for language tasks. We start by applying the approach of to neural machine translation (NMT) with sequence-to-sequence BID25 as an underlying model. Our goal is to find new recurrent cells that can work better than Long Short-term Memory (LSTM) BID6. We then introduce a novel "stack" search space as an alternative to the fixed-structure tree search space defined in. We use this new search space to find similarity functions for the attention mechanism in NMT BID0 BID9. Through our extensive searches across two translation benchmarks, small IWSLT English-Vietnamse and large WMT German-English, we report challenges in performing cell searches for NMT and demonstrate initial success on attention searches with translation improvements over strong baselines. Lastly, we show that the attention similarity functions found for NMT are transferable to the reading comprehension task on the Stanford Question Answering Dataset (SQuAD) BID14, yielding non-trivial improvements over the standard dot-product function. Directly running NAS attention search on SQuAD boosts the performance even further. Figure 1: Tree search space for recurrent cells -shown is an illustration of a tree search space specifically designed for searching over LSTM-inspired cells. The figure was obtained from with permission. Left: the tree that defines the computation steps to be predicted by controller. Center: an example set of predictions made by the controller for each computation step in the tree. Right: the computation graph of the recurrent cell constructed from example predictions of the controller. In neural architecture search (NAS), a controller iteratively samples a model architecture which is then run against a task of interest to obtain a reward signal. The reward signal is then used to update the controller so as to produce better and better architectures over time. We follow the architecture search setup in, which was originally developed for a small language modeling task on PTB, and adapt it to other language tasks. We first focus on translation and conduct searches on both small and large scale translation tasks (IWSLT and WMT respectively, with details in Section 3). In terms of reward functions, we have a choice of using either perplexity or BLEU scores which have been known to be well-correlated in neural machine translation BID9. Formally, we scale the reward scores to be within as follows 1: DISPLAYFORM0 We now describe the search spaces over recurrent cells and attention similarity functions. This search space is identical to what was designed by to search for LSTMinspired cells. Each instance of the search space takes as inputs (state h t−1, cell c t−1, input x t) and produces as outputs (state h t, cell c t). The idea is to define computation through a balanced binary tree which we start off first by connecting each leaf node with inputs (h t−1, x t) and producing output h t at the top node. The left part of Figure 1 illustrates a binary computation tree of depth 2. The RNN controller will then decide for each node in the tree what combined operations, e.g., add or mul, to use and what nonlinearity, e.g., tanh or sigmoid, to be immediately followed. The controller will also decide how to incorporate ("cell inject") the previous cell c t−1 and which nodes ("cell indices") in the tree to use for that injection as well as the output of the new cell. The RNN controller is illustrated in the center part of Figure 1 and the realization of an LSTM-inspired instance is shown on the right part. The parameters of instance is defined by linear transformation of each inputs (h t−1, x t) before passing to each leaf node. LSTM-inspired Cells for NMT In this work, we are interested in how this search space, which works well for the language modeling task, performs on translation. Our set of combined operations are element-wise addition and multiplication; whereas the set of non-linear functions are (identity, tanh, sigmoid, relu). We use a binary tree of depth 4 with 8 leaf nodes. We propose a simple stack-based programming language as an alternative to the fixed structure of the previous tree search space. This is reminiscent of the Push language which is historically used in genetic programming BID24. A program consists of a sequence of function calls, with each function popping N arguments from the top of the stack and pushing M back onto it. As illustrated in Figure 2, we consider in this work only unary or binary ops, i.e., N = 1 or 2, with M = 1. If there are not enough arguments in the stack, such as a binary operation when there is a single item in the stack, the operation is ignored. This ensures every program is valid without additional constraints on the controller. The program is also given the ability to copy outputs produced within S steps ago. Such capability is achieved through ops (copy 0, . . ., copy S−1) which the controller can predict together with other unary and binary ops. The indices in Figure 2 indicate the order in which outputs were generated so copy ops can be applied. At the start of execution, input arguments of the same shape and data type are both pushed onto the stack. At the end of execution, we either sum all remaining values or take the top value of the stack as the output. DISPLAYFORM0 Figure 2: General stack search space -shown is an illustration of how the stack changes over a sequence of operations. In this example, the controller predicts (copy 0, linear, sigmoid, add) as a sequence of operations to be applied. The stack has a vector x as an initial input and produces x + σ (W x) as an output. Attention Search for NMT As attention mechanism is key to the success of many sequence-tosequence models, including neural machine translation BID0, it is worthwhile to improve it. To recap, we show in Figure 3 an instance of the attention mechanism proposed by BID8 for NMT. At each time step, the current target hidden state h t (a "query") is used to compare with each of the source hidden statesh s (memory "keys") to compute attention weights which indicate which memory slots are most useful for the translation at that time step. The attention weight for each source position s is often computed as exp score(ht,hs) DISPLAYFORM1, with S being the source length. In BID8, various forms of the scoring function have been proposed, e.g., the simple dot-producth s h t or the bilinear formh s W h t. Figure 3: Attention Mechanism -example of an attention-based NMT system as described in BID8. We highlight in detail the first step of the attention computation. Instead of hand-designing these functions, we propose to search through the set of scoring functions score(q, k) using our stack-based search space. The stack starts out with a key vector k followed by a query vectory q on top. The RNN controller predicts a list of L ops, followed by a special reduce op that is used to turn vectors into scalars by summing over the final dimension as illustrated in Figure 4. L is the program length, which controls the search space complexity and is set to 8 in our experiments. The set of binary and unary ops include (linear, mul, add, sigmoid, tanh, relu, identity) with mul being element-wise multiplication. The occurrence of op identity is for the controller to shorten the program as needed. DISPLAYFORM2 k Figure 4: Stack Search Space for Attention -shown is the bilinear scoring function BID8 as an instance of the stack search space for attention. The controller predicts (linear, mul) as ops. All scoring functions end with a reduce op that turns vectors into scalars. Attention mechanisms are a core part of modern question answering systems. In the extractive problem setting, given a query sentence and a context paragraph, the task is to output start and end positions in the text, called a span, which contains the answer. On the most common dataset of this type, Stanford Question Answering Dataset (SQuAD) BID13, the top performing models all perform some variant of attention between the encoded query sentence and encoded context paragraph. Many of the best models are variants of a model known as Bidirectional Attentive Flow (BiDAF) BID21. BiDAF encodes the query and context separately, then performs bidirectional attention from the query vectors over the context vectors and from the context vectors over the query vectors. The similarity is generally computed as the dot product between the two vectors. To test the generalization of our attention functions and search, we apply the best attention mechanisms from NMT in place of the dot product. We also repeat the search procedure directly on the SQuAD dataset. The search is similar as for NMT, with the addition of a unary separable 1D convolution operator. We consider two translation setups, small and large-scale ones, to test different effects of Neural Architecture Search (NAS). Remember that, in NAS, there are often two phases involved: (a) search architectures -in which we train many child models to find best architectures according to the reward signals and (b) run convergence -where we take top architectures found and train full models until convergence. To make things clear, for each of the translation setup below, we will describe the data used, as well as the training hyperparameters for both the child and the full models. Our evaluation metrics include both perplexity and BLEU BID11 scores; whereas, the NAS reward signals can be either perplexity or BLEU, which we will detail later. Data We utilize a small parallel corpus of English-Vietnamese TED talks (133K sentence pairs), provided by the IWSLT Evaluation Campaign BID4. Following BID7, we tokenize with the default Moses tokenizer and replace words whose frequencies are less than 5 by <unk>. 2 The final data has vocabulary sizes of 17K for English and 7.7K for Vietnamese. We use the TED tst2012 (1553 sentences) as a validation set for hyperparameter tuning and TED tst2013 (1268 sentences) as a test set. Full-model hyperparameters Each full model is an attention-based sequence-to-sequence model with 2 LSTM layers of 512 units each; the encoder is bidirectional and the embedding dimension is also 512. We train each child model for 12K steps (roughly 12 epochs) with dropout rate of 0.2 (i.e., keep probability 0.8) and batch size 128. Our optimizer is SGD with learning rate 1; after 8K steps, we start halving learning rate every 1K step. We use 5 for gradient norm clipping and uniformly initialize the parameters within [−0.1, 0.1]. The exact implementation and setup were obtained from the NMT tutorial BID10 ).Child-model hyperparameters Since this dataset is small, the hyperparameters of the child models for NAS are identical to those of the full models. Data We consider the WMT German-English translation task with 4.5M training sentence pairs. The data is split into subword units using the BPE scheme BID20 with 32K operations. We use newstest2013 (3000 sentences) as a development set and report translation performances on both newstest2014 (2737 sentences) and newstest2015 (2169 sentences). Full-model hyperparameters We train strong translation models based on the architecture of Google's Neural Machine Translation systems BID27 ) with a implementation in BID10. The model consists of 4 LSTM layers of 1024 units (the encoder starts with a bidirectional LSTM followed by three unidirectional layers); embedding dim is 1024. The hyperparameters are similar to those of the English-Vietnamese setup: init range [−0.1, 0.1], dropout 0.2, gradient norm 5.0. We train with SGD for 340K steps (10 epochs). The learning rate is 1.0; after 170K steps, we start halving learning rate every 17K step. Child-model hyperparameters Since we cannot afford to run NAS on full models, our child model is a scaled-down version of the full one with 2 layers and 256 units, trained for 10K steps. Following, we train the controller RNN using the Proximal Policy Optimization BID19 with a learning rate of 0.0005. To encourage exploration, an entropy penalty 0.0001 was used. We use an exponential moving average of previous rewards with weight 0.99 as a baseline function. The controller weights are uniformly initialized within [−0.1, 0.1]. We use minibatches of 20 architectures to update the controller RNN weights. During search, we employ a global workqueue system similar to to process a pool of child networks proposed by the RNN controller. In our experiments, the pool of workers consists of 100-200 GPUs. We stop the searches once the dev performance saturates and the set of unique child models remains unchanged. Once the search is over, the top 10 architectures are then chosen to train until convergence. We present in Table 1 of neural architecture search for translation. We compare over strong baselines provided by BID10, which replicate Goole's NMT architectures BID27. As we can see in the first three rows, the strong baseline trained with SGD and LSTM as the basic unit outperforms NASCell, the TensorFlow public implementation of the best recurrent cell found on the PTB language modeling task by. 4 Architecture searches directly on translation tasks yield better performances compared to NASCell. We can find cells that outperform the baseline in the IWSLT benchmark with 26.2 BLEU. Beating the strong baseline on the larger WMT task remains a challenge; with cell searches performed directly on WMT, we can narrow the gap with the baseline, achieving 28.4 BLEU. Finally, by performing attention searches on WMT, we were able to outperform the WMT baseline with 29.1 BLEU. The same attention function found is also transferable to the small IWSLT benchmark, yielding a high score of 26.0 BLEU. IWSLT (small) WMT (large) tst2013 newstest2015 Google NMT baseline [adam] BID10 21.8 26.8 Google NMT baseline [sgd] BID10 25.5 28.8 NASCell on LM [sgd] 25.1 27.7 NASCell on IWSLT [ppl, sgd] Table 1: Neural architectural searches for translation -shown are translation performances in BLEU for various neural architecture searches (NAS) at the cell and attention levels. Searches are performed on either the small IWSLT or large WMT translation setups with reward functions being either ppl (perplexity) or BLEU. For each NAS search, we report on both translation setups. For NASCell, we use the TensorFlow public implementation by and run on our translation setups. We highlight in bold numbers that are best in each group. In this section, we continue in Table 1 to further discuss the effects of optimizers and reward functions used in architecture search. We also show the top attention functions found by NAS and their effects. Lastly, we examine the transferability of these attention functions and searches to the task of reading comprehension. We found that optimizers used for NMT models matter greatly in architecture search. From the training plots in FIG2, it seems to appear that Adam outperforms SGD greatly, achieving much higher BLEU scores on both the dev and test sets after a fixed training duration of 10K steps per child model. However, as observed in rows 5 and 6 of Table 1, recurrent cells found by Adam are unstable, yielding much worse performance compared to those found by SGD. We have tried using Glorot initialization scheme BID5 but could not alleviate the problem of large gradients when using Adam-found cells. We suspect further hyperparameter tuning for final-model training might help. We also carry a small experiment comparing the reward functions described in Eq for the attention search. From Figure 6, the reward function based on BLEU trivially leads to higher dev and test BLEU scores during the search. The attention functions found does transfer to higher BLEU scores as shown in row 8 of Table 1. What surprised us was the fact that the attention mechanisms found with perplexity-based reward function perform poorly. The top-performing attention similarity functions found are: reduce(sigmoid(relu(tanh(W (k q))))) and reduce(sigmoid(relu(W (k q)))). At first, the equations look puzzling with multiple nonlinearity functions stacked together, which we think due to noise in the design of the search space that lets the controller favor over nonlinear functions. However, a closer look does reveal an interesting pattern that keys and queries are encouraged to interact element-wise, followed by linear transformation, nonlinearity, before the final reduce-sum op. On the other hand, several bad attention functions have the following pattern of applying nonlinearity immediately after elementwise multiplication, e.g., reduce(sigmoid(W (tanh(tanh(k q))))). Nevertheless, we think the search space could have been improved by predicting when a program can end and when to perform reduce operations. Figure 6: Effects of reward functions for attention searches -shown are similar plots to those in FIG2 with two attention searches that differ in terms of reward functions used: one based one BLEU while the other is based on perplexity (pp). For brevity, we only show plot for dev BLEU and total entropy. SQuAD Systems F 1 BiDAF BID22 77.3 Our baseline (dot-product) 80.1 NASAttention on NMT 80.5 NASAttention on SQuAD 81.1 For the reading comprehension setting, we evaluate on the Stanford Question Answering dataset as discussed in 2.3. We report in F 1 which measures the portion of overlap tokens between the predicted answer and groundtruth. Model Our model details are as follows: we embed both the query and context with pretrained GLoVE BID12 embeddings. Each is then encoded independently using the embedding encoder, sharing weights between the two. We then combine them with context-toquery attention (leaving out the query-to-context attention from BiDAF), with the output from each position in the context the of attending over the query with that position's encoding. When doing attention search, we search over the form of the equation to compute this similarity. Finally, we take the output of this attention mechanism and run through a stack of three model encoder, giving three outputs x 1, x 2, and x 3, each of which is the length of the context sentence. The probability of the span starting at each position is computed as W 0 [x 1, x 2], and the probability of it being the end position is W 1 [x 1, x 3]. We score each span as the product of its start position probability and end position probability, returning the span with the highest score. Results TAB1 demonstrates that the NASAttention found with NMT does provide improvement over the baseline with dot-product attention, yielding a F 1 score of 80.5. When performing attention search directly on SQuAD, the performance is further boosted to 80.1 F 1. We find that the best performing attention function for the context-to-query attention is simple yet novel: f (key, query) = relu(conv(key • query)), where conv is a 1d separable convolution with a kernel size of 3. Neighboring positions in the convolution correspond to neighboring keys (in this case in the encoded question) being queried with the same query vector. In this paper, we have made a contribution towards extending the success of neural architecture search (NAS) from vision to another domain, languages. Specifically, we are first to apply NAS to the tasks of machine translation and reading comprehension at scale. Our newly-found recurrent cells perform better on translation than previously-discovered NASCell. Furthermore, we propose a novel stack-based search space as a more flexible alternative to the fixed-structure tree search space used for recurrent cell search. With this search space, we find new attention functions that outperform strong translation baselines. In addition, we demonstrate that the attention search are transferable to the SQuAD reading comprehension task, yielding nontrivial improvements over dot-product attention. Directly running NAS attention search on SQuAD boosts the performance even further. We hope that our extensive experiments will pave way for future research in NAS for languages.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We explore neural architecture search for language tasks. Recurrent cell search is challenging for NMT, but attention mechanism search works. The result of attention search on translation is transferable to reading comprehension.

represent the tldr for finding the corresponding scientific article:

For example, 'scale generative classifiers  on complex datasets, and evaluate their effectiveness to reject illegal inputs including out-of-distribution samples and adversarial examples.' should be close to 'Deep Infomax~(DIM) is an unsupervised representation learning framework by maximizing the mutual information between the inputs and the outputs of an encoder, while probabilistic constraints are imposed on the outputs. In this paper, we propose Supervised Deep InfoMax~(SDIM), which introduces supervised probabilistic constraints to the encoder outputs. The supervised probabilistic constraints are equivalent to a generative classifier on high-level data representations, where class conditional log-likelihoods of samples can be evaluated. Unlike other works building generative classifiers with conditional generative models, SDIMs scale on complex datasets, and can achieve comparable performance with discriminative counterparts. With SDIM, we could perform \emph{classification with rejection}. Instead of always reporting a class label, SDIM only makes predictions when test samples' largest logits surpass some pre-chosen thresholds, otherwise they will be deemed as out of the data distributions, and be rejected. Our experiments show that SDIM with rejection policy can effectively reject illegal inputs including out-of-distribution samples and adversarial examples. Non-robustness of neural network models emerges as a pressing concern since they are observed to be vulnerable to adversarial examples . Many attack methods have been developed to find imperceptible perturbations to fool the target classifiers (; ;). Meanwhile, many defense schemes have also been proposed to improve the robustnesses of the target models (; Tramèr et al., 2017; ;). An important fact about these works is that they focus on discriminative classifiers, which directly model the conditional probabilities of labels given samples. Another promising direction, which is almost neglected so far, is to explore robustness of generative classifiers . A generative classifier explicitly model conditional distributions of inputs given the class labels. During inference, it evaluates all the class conditional likelihoods of the test input, and outputs the class label corresponding to the maximum. Conditional generative models are powerful and natural choices to model the class conditional distributions, but they suffer from two big problems: it is hard to scale generative classifiers on high-dimensional tasks, like natural images classification, with comparable performance to the discriminative counterparts. Though generative classifiers have shown promising of adversarial robustness, they hardly achieve acceptable classification performance even on CIFAR10; ). The behaviors of likelihood-based generative models can be counter-intuitive and brittle. They may assign surprisingly higher likelihoods to out-of-distribution (OoD) samples . discuss the issues of likelihood as a metric for density modeling, which may be the reason of non-robust classification, e.g. OoD samples detection. In this paper, we propose supervised deep infomax (SDIM) by introducing supervised statistical constraints into deep infomax , an unsupervised learning framework by maximizing the mutual information between representations and data. SDIM is trained by optimizing two objectives: maximizing the mutual information (MI) between the inputs and the high-level data representations from encoder; ensuring that the representations satisfy the supervised statistical constraints. The supervised statistical constraints can be interpreted as a generative classifier on high-level data representations giving up the full generative process. Unlike full generative models making implicit manifold assumptions, the supervised statistical constraints of SDIM serve as explicit enforcement of manifold assumption: data representations (low-dimensional) are trained to form clusters corresponding to their class labels. With SDIM, we could perform classification with rejection . SDIMs reject illegal inputs based on off-manifold conjecture , where illegal inputs, e.g. adversarial examples, lie far away from the data manifold. Samples whose class conditionals are smaller than the pre-chosen thresholds will be deemed as off-manifold, and prediction requests on them will be rejected. The contributions of this paper are: • We propose Supervised Deep Infomax (SDIM), an end-to-end framework whose probabilistic constraints are equivalent to a generative classifier. SDIMs can achieve comparable classification performance with similar discrinimative counterparts at the cost of small over-parameterization. • We propose a simple but novel rejection policy based on off-manifold conjecture: SDIM outputs a class label only if the test sample's largest class conditional surpasses the prechosen class threshold, otherwise outputs rejection. The choice of thresholds relies only on training set, and takes no additional computations. • Experiments show that SDIM with rejection policy can effectively reject illegal inputs, including OoD samples and adversarial examples generated by a comprehensive group of adversarial attacks. Deep InfoMax is an unsupervised representation learning framework by maximizing the mutual information (MI) of the inputs and outputs of an encoder. The computation of MI takes only input-output pairs with the deep neural networks based esimator MINE . Let E φ be an encoder parameterized by φ, working on the training set, and generating. DIM is trained to find the set of parameters φ such that: the mutual information I(X, Y) is maximized over sample sets X and Y. the representations, depending on the potential downstream tasks, match some prior distribution. Denote J and M the joint and product of marginals of random variables X, Y respectively. MINE estimates a lower-bound of MI with Donsker-Varadhan representation of KL-divergence: where T ω (x, y) ∈ R is a family of functions with parameters ω represented by a neural network. Since in representation learning we are more interested in maximizing MI, than its exact value, non-KL divergences are also favorable candidates. We can get a family of variational lower-bounds using f -divergence representations : where f * is the Fenchel conjugate of a specific divergence f. For KL-divergence, f * (t) = e (t−1). A full f * list is provided in Tab. 6 of. Noise-Contrastive Estimation (Gutmann & Hyvärinen, 2010) can also be used as lower-bound of MI in "infoNCE" . All the components of SDIM framework are summurized in Fig. 1. The focus of Supervised Deep InfoMax (SDIM) is on introducing supervision to probabilistic constraints of DIM for (generative) classification. We choose to maximize the local MI, which has shown to be more effective in classification tasks than maximizing global MI. Equivalently, we minimize J MI: where L φ (x) is a local M × M feature map of x extracted from some intermediate layer of encoder E, andĨ can be any possible MI lower-bounds. By adopting a generative approach p(x, y) = p(y)p(x|y), we assume that the data follows the manifold assumption: the (high-dimensional) data lies on low-dimensional manifolds corresponding to their class labels. Denotex the compact representation generated with encoder E φ (x). In order to explicitly enforce the manifold assumption, we admit the existence of data manifold in the representation space. Assume that y is a discrete random variable representing class labels, and p(x|y) is the real class conditional distribution of the data manifold given y. Let p θ (x|y) be the class conditionals we model parameterized with θ. We approximate p(x|y) by minimizing the KL-divergence between p(x|y) and our model p θ (x|y), which is given by: where the first item on RHS is a constant independent of the model parameters θ. Eq. 4 equals to maximize the expectation Ex,y∼p(x,y) [log p θ (x|y)]. In practice, we minimize the following loss J NLL, equivalent to empicically maximize the above expectation over: Besides the introduction of supervision, SDIM differs from DIM in its way of enforcing the statistical constraints: DIM use adversarial learning to push the representations to the desired priors, while SDIM directly maximizes the parameterized class conditional probability. Maximize Likelihood Margins Since a generative classifier, at inference, decides which class a test input x belongs to according to its class conditional probability. On one hand, we maximize samples' true class conditional probabilities (classes they belong to) using J NLL; On the other hand, we also hope that samples' false class conditional probabilities (classes they do not belong to) can be minimized. This is assured by the following likelihood margin loss J LM: where K is a positive constant to control the margin. For each encoder outputx i, the C − 1 truefalse class conditional gaps are squared 1, which quadratically increases the penalties when the gap becomes large, then are averaged. Putting all these together, the complete loss function we minimize is: Parameterization of Class Conditional Probability Each of the class conditional distribution is represented as an isotropic Gaussian. So the generative classifier is simply a embedding layer with C entries, and each entry contains the trainable mean and variance of a Gaussian. This minimized parameterization encourages the encoder to learn simple and stable low-dimensional representations that can be easily explained by even unimodal distributions. Considering that we maximize the true class conditional probability, and minimize the false class conditional probability at the same time, we do not choose conditional normalizing flows, since the parameters are shared across class labels, and the training can be very difficult. , each class conditional probability is represented with a VAE, thus scaling to complex datasets with huge number of classes, e.g. ImageNet, is almost impossible. A generative approach models the class-conditional distributions p(x|y), as well as the class priors p(y). For classification, we compute the posterior probabilities p(y|x) through Bayes' rule: The prior p(y) can be computed from the training set, or we simply use uniform class prior for all class labels by default. Then the prediction of test sample x * from posteriors is: The drawback of the above decision function is that it always gives a prediction even for illegal inputs. Instead of simply outputting the class label that maximizes class conditional probability of x *, we set a threshold for each class conditional probability, and define our decision function with rejection to be: The model gives a rejection when log p(x * |y *) is smaller than the threshold δ y *. Note that here we can use p(x * |y *) and p(x * |y *) interchangeably. This is also known as selective classification or classification with reject option (See Supp. A) 4 RELATED WORKS Robustness of Likelihood-based Generative Models Though likelihood-based generative models have achieved great success in samples synthesis, the behaviors of their likelihoods can be counter-intuitive. Flow-based models and as well as VAEs , surprisingly assign even higher likelihoods to out-of-distribution samples than the samples in the training set . Pixel-level statistical analyses in show that OoD dataset may "sit inside of" the in-distribution dataset (i.e. with roughly the same mean but smaller variance). observe that adversarial examples are outside the training distribution via statistical testing. DefenseGAN models real data distribution with the generator G of GAN. At inference, instead of feeding the test input x to the target classifier directly, it searches for the "closest" sample G(z *) from generator distribution to x as the final input to the classifier. It ensures that the classifier only make predictions on the data manifold represented by the generator, ruling out the potential adversarial perturbations in x. PixelDefend takes a similar approach which uses likelihood-based generative model -PixelCNN to model the data distribution. Both DefenseGAN and PixelDefend are additionally trained as peripheral defense schemes agnostic to the target classifiers. Training generative models on complex datasets notoriously takes huge amount of computational resources . In contrast, the training of SDIM is computationally similar to its discriminative counterpart. The verification of whether inputs are offmanifold is a built-in property of the SDIM generative classifier. The class conditionals of SDIM are modeled on low-dimensional data representations with simple Gaussians, which is much easier, and incurs very small computations. Datasets We evaluate the effectiveness of the rejection policy of SDIM on four image datasets: MNIST, FashionMNIST (both resized to 32×32 from 28×28); and CIFAR10, SVHN. See App. B.1 for details of data processing. For out-of-distribution samples detection, we use the dataset pairs on which likelihood-based generative models fail : FashionMNIST (in)-MNIST (out) and CIFAR10 (in)-SVHN (out). Adversarial examples detection are evaluated on MNIST and CIFAR10. Choice of thresholds It is natural that choosing thresholds based on what the model knows, i.e. training set, and can reject what the model does not know, i.e. possible illegal inputs. We set one threshold for each class conditional. For each class conditional probability, we choose to evaluate on two different thresholds: 1st and 2nd percentiles of class conditional log-likelihoods of the correctly classified training samples. Compared to the detection methods proposed in, our choice of thresholds is much simpler, and takes no additional computations. Models A typical SDIM instance consists of three networks: an encoder, parameterized by φ, which outputs a d-dimensional representation; mutual information evaluation networks, i.e. T ω in Eqn. and Eqn.; and C-way class conditional embedding layer, parameterized by θ, with each entry a 2d-dimensional vector. We set d = 64 in all our experiments. For encoder of SDIM, we use ResNet on 32 × 32 with a stack of 8n + 2 layers, and 4 filter sizes {32, 64, 128, 256}. The architecture is summarized as: The last layer of encoder is a d-way fully-connected layer. To construct a discriminative counterpart, we simply set the output size of the encoder's last layer to C for classification. We use ResNet10 (n = 1) on MNIST, FashionMNIST, and ResNet26 (n = 3) on CIFAR10, SVHN. We report the classification accuracies (see Tab. 1) of SDIMs and the discriminative counterparts on clean test sets. Results show that SDIMs achieve the same level of accuracy as the discriminative counterparts with slightly increased number of parameters (17% increase for ResNet10, and 5% increase for ResNet26). We are aware of the existence of better reported on these datasets using more complex models or automatically designed architectures , but pushing the state-of-the-art is not the focus of this paper. , both model class conditional probability with VAE , and achieve acceptable accuracies (> 98%) on MNIST. However, it is hard for fully conditional generative models to achieve satisfactory classification accuracies even on CIFAR10. On CIFAR10, methods in achieve only < 50% accuracy. They also point out that the classification accuracy of a conditional PixelCNN++ is only 72.4%. The test accuracy of ABS in is only 54%. In contrast, SDIM could achieve almost the same performance with similar discriminative classifier by giving up the full generative process, and building generative classifier on high-level representations. improves the accuracy to 92% by feeding the features learned by powerful discriminative classifier-VGG16 to their generative classifiers, which also suggests that modeling likelihood on high-level representation (features) is more favorable for generative classification than pixel-level likelihood of fully generative classifiers. For classification tasks, discovering discriminative features is much more important than reconstructing the all the image pixels. Thus performing generative classification with full generative models may not be the right choice. We also investigate the implications of the proposed decision function with rejection under different thresholds. The in Tab. 2 show that choosing a higher percentile as threshold will reject more prediction requests. At the same time, the classification accuracies of SDIM on the left test sets become increasingly better. This demonstrate that out rejection policy tend to reject the ones on which SDIMs make wrong predictions. Table 2: Classification performances of SDIMs using the proposed decision function with rejection. We report the rejection rates of the test sets and the accuracies on the left test sets for each threshold. Class-wise OoD detections are performed, and mean detection rates over all in-distribution classes are reported in Tab. 3. For each in-distribution class c, we evaluate the log-likelihoods of the whole OoD dataset. Samples whose log-likelihoods are lower the class threshold δ c will be detected as OoD samples. Same evaluations are applied on conditional Glows with 10th percentile thresholds, but the are not good. The are clear and confirm that SDIMs, generative classifiers on high-level representations, are more effective on classification tasks than fully conditional generative models on raw pixels. Note that fully generative models including VAE used in; fail on OoD detection. The stark difference between SDIM and full generative models (flows or VAEs) is that SDIM models samples' likelihoods in the high-level representation spaces, while generative models evaluate directly on the raw pixels. See Supp. C for more about the histograms of the class conditionals of in-out distributions. Table 3: Mean detection rates of SDIMs and Glows with different thresholds on OoD detection. We comprehensively evaluate the robustness of SDIMs against various attacks: • gradient-based attacks: one-step gradient attack FGSM , its iterative variant projected gradient descent , CW-L 2 attack , deepfool . • score-based attacks: local search attack . • decision-based attack: boundary attack . Attacks Using Cross-Entropy We find that SDIMs are much more robust to gradient-based attacks using cross-entropy, e.g. FGSM and PGD, since the gradients numerically vanish as a side effect of the likelihood margin loss J LM of SDIM. This phenomenon is similar to some defences that try to hinder generations of adversarial examples by masking the gradients on inputs. While full generative classifiers in still suffer from these attacks. See Supp. D.1 for detailed . Conservative Adversarial Examples Adversarial attacks aim to find the minimal perturbations that sufficiently change the classification labels, i.e. flip other logits to be the largest one. We show case examples on MNIST generated by untargeted attacks and their logits in Tab. 4 (See Supp. D.2 for examples of CIFAR10). Though these attacks successfully flip the logits, they are designed to be conservative to avoid more distortions to the original images. As a , the largest logits of adversarial examples are still much lower than the thresholds, so they can be detected by our rejection policy. We find that our rejection policy performs perfectly on MNIST, but fails to detect all adversarial examples on CIFAR10 except for Boundary attack (See Tab. 5). It seems to be a well-known observation that models trained on CIFAR10 are more vulnerable than one trained on MNIST. connects this observation to the generalization of models. They found that many test samples, though correctly classified, are close to the misclassfied samples, which implies the existence of adversarial examples. If a model has higher error rate, it would take smaller perturbations to move correctly classified samples to misclassified areas. Table 4: Full logits of the adversarial examples generated with different attacks. The original image is the first sample of class 0 of MNIST test set. The first row gives the 1st percentile thresholds, and the second row shows the logits of the original image. The largest logits are marked in bold. Adversarial examples with more confidence Based on the observations above, a natural question we should ask is: can we generate adversarial examples with not only successfully flipped logits, Table 5: Detection rates of our rejection policies. We perform untargeted adversarial evaluation on the first 1000 images of test sets. CW-L 2 is not involved here, but carefully investigated below. but also the largest logit larger than some threshold value? Unlike the conservativeness on paying more distortions of other attacks, CW attack allows us to control the gap between largest and second largest logits with some confidence value κ. We perform targeted CW attacks with confidences κ = {0, 500, 1000} (Tab. 6). We find that increasing the confidences help increasing the largest logits of adversarial examples to some extent, but may lead to failures of generation. The sensitivity to confidence values is also different given different targets. The success rates of generating adversarial examples monotonically decreases with the confidences increasing (Tab. 7). Note that on discriminative counterparts, CW-L 2 with the same settings easily achieves 100% success rates. This means that explicitly forcing data representations to form clusters with maximum margins between them help increase average distances between normal samples and the nearest misclassified areas, thus increase the hardness of finding minimal adversarial perturbations. In this case, it takes a large enough adversarial perturbation to move a sample from its cluster to the other. Meanwhile, detection rates remain satisfactory on MNIST, but obviously decline on CIFAR10. For victim generative classifiers in under CW-L 2 attack, the detection rates of adversarial examples using the proposed detection methods can be > 95% on MNIST, but fall < 50% on even CIFAR10-binary (their models don't scale on CIFAR10, and CW-L 2 with non-zero confidences are also not evaluated). Table 7: Targeted adversarial evaluations of our rejection policies on the first 1000 test samples. We report the detection rates with different thresholds and success rates of generating adversarial examples. Discussions on off-manifold conjecture challenges whether the off-manifold conjecture holds in general. They experiment on synthetic dataset-two high-dimensional concentric spheres with theoretical analyses, showing that even for a trained classifier with close to zero test error, there may be a constant fraction of the data manifold misclassified, which indicates the existence of adversarial examples within the manifold. But there are still several concerns to be addressed: First, as also pointed out by the authors, the manifolds in natural datasets can be quite complex than that of simple synthesized dataset. draws similar from analyses on synthesized data with particular geometry. So the big concern is whether the in; still hold for the manifolds in natural datasets. A practical obstacle to verify this is that works modeling the full generative processes are based on manifold assumption, but provide no explicit manifolds for analytical analyses like;. While SDIM enables explicit and customized manifolds on high-level data representations via probabilistic constraints, thus enables analytical analyses. In this paper, samples of different classes are trained to form isotropic Gaussians corresponding to their classes in representation space (other choices are possible). The relation between the adversarial robustness and the forms and dimensionalities of data manifolds is to be explored. Second, in their experiments, all models evaluated are discriminative classifiers. Considering the recent promising of generative classifiers against adversarial examples, would using generative classifiers lead to different ? One thing making us feel optimistic is that even though the existence of adversarial examples is inevitable, suggest that adversarial robustness can be improved by minimizing the test errors, which is also supported by our experimental differences on MNIST and CIFAR10. We introduce supervised probabilistic constraints to DIM. Giving up the full generative process, SDIMs are equivalent to generative classifiers on high-level data representations. Unlike full conditional generative models which achieve poor classification performance even on CIFAR10, SDIMs attain comparable performance as the discriminative counterparts on complex datasets. The training of SDIM is also computationally similar to discriminative classifiers, and does not require prohibitive computational resources. Our proposed rejection policy based on off-manifold conjecture, a built-in property of SDIM, can effectively reject illegal inputs including OoD samples and adversarial examples. We demonstrate that likelihoods modeled on high-level data representations, rather than raw pixel intensities, are more robust on downstream tasks without the requirement of generating real samples. We make comparisons between SDIM and GBZ, which consistently performs best in Deep Bayes. FGSM and PGD-L ∞ The in Fig 4 and Fig 5 show that SDIM performs consistently better than the baseline. We find that increasing the distortion factor of FGSM has no influences of SDIM's accuracy, and the adversarial examples keep the same. Recall that the class conditionals are optimized to keep a considerable margin. Before evaluating the cross entropy loss, softmax is applied on the class conditionals log p(x|c) to generate a even sharper distribution. So for the samples that are correctly classified, their losses are numerically zeros, and the gradient on inputs ∇J x (x, y) are also numerically zeros. The PGD-L ∞ we use here is the randomized version 2, which adds a small random perturbation before the iterative loop. The randomness is originally introduced to generate different adversarial examples for adversarial training, but here it breaks the zero loss so that the gradient on inputs ∇J x (x, y) will not be zeros in the loop. FGSM can also be randomized (Tramèr et al., 2017), which can be seen as a one-step variant of randomized PGD. This phenomena is similar to what some defenses using gradient obfuscation want to achieve. Defensive distillation masks the gradients of cross-entropy by increasing the temperature of softmax. But for CW attacks, which do not use cross-entropy, and operate on logits directly, this could be ineffective. 4 -353.1 -471.6 -400.3 -342.7 -367.2 -486.4 -326.4 Boundary 213.9 -417.4 -458.3 -548.0 -587.4 -236.3 214.0 -1246.1 -171.2 -555.6 LocalSearch 165.2 -485.7 190.9 -325.6 -439.0 -379.0 -318.8 -327.5 -357.9 -272.3 Table 9: Full logits of adversarial examples generated with different attacks. The original image is the fist sample of class 0 of CIFAR10 test set. The first row gives the 1st percentile thresholds, and the second row shows the logits of the original image. The largest logits are marked in bold.'

The paper introduces a method of training generative recurrent networks that helps to plan ahead. We run a second RNN in a reverse direction and make a soft constraint between cotemporal forward and backward states.