Question: Represent the text to find a very short summary (1-2 sentences max):

Question: Deep neural networks (DNNs) have witnessed as a powerful approach in this year by solving long-standing Artificial intelligence (AI) supervised and unsupervised tasks exists in natural language processing, speech processing, computer vision and others. In this paper, we attempt to apply DNNs on three different cyber security use cases: Android malware classification, incident detection and fraud detection. The data set of each use case contains real known benign and malicious activities samples. These use cases are part of Cybersecurity Data Mining Competition (CDMC) 2017. The efficient network architecture for DNNs is chosen by conducting various trails of experiments for network parameters and network structures. The experiments of such chosen efficient configurations of DNNs are run up to 1000 epochs with learning rate set in the range [0.01-0.5]. Experiments of DNNs performed well in comparison to the classical machine learning algorithm in all cases of experiments of cyber security use cases. This is due to the fact that DNNs implicitly extract and build better features, identifies the characteristics of the data that lead to better accuracy. The best accuracy obtained by DNNs and XGBoost on Android malware classification 0.940 and 0.741, incident detection 1.00 and 0.997, and fraud detection 0.972 and 0.916 respectively. The accuracy obtained by DNNs varies -0.05%, +0.02%, -0.01% from the top scored system in CDMC 2017 tasks. In this era of technical modernization, explosion of new opportunities and efficient potential resources for organizations have emerged but at the same time these technologies have ed in threats to the economy. In such a scenario proper security measures plays a major role. Now days, hacking has become a common practice in organizations in order to steal data and information. This highlights the need for an efficient system to detect and prevent the fraudulent activities. cyber security is all about the protection of systems, networks and data in the cyberspace. Malware remains one of the maximum enormous security threats on the Internet. Malware are the softwares which indicate malicious activity of the file or programs. These are unwanted programs since they cause harm to the intended use of the system by making it behave in a very different manner than it is supposed to behave. Solutions with Antivirus and blacklists are used as the primary weapons of resistance against these malwares. Both approaches are not effective. This can only be used as an initial shelter in real time malware detection system. This is primarily due to the fact that both approaches completely fails in detecting the new malware that is created using polymorphic, metamorphic, domain flux and IP flux. Machine learning algorithms have played a pivotal role in several use cases of cyber security BID0. Fortunately, deep learning approaches are prevailing subject in recent days due to the remarkable performance in various long-standing artificial intelligence (AI) supervised and unsupervised challenges BID1. This paper evaluates the effectiveness of deep neural networks (DNNs) for cyber security use cases: Android malware classification, incident detection and fraud detection. The paper is structured as follows. Section II discusses the related work. Section III discusses the knowledge of deep neural networks (DNNs). Section IV presents the proposed methodology including the description of the data set. Results are displayed in Section V. Conclusion is placed in Section VI. This section discusses the related work for cyber security use cases: Android malware classification, incident detection and fraud detection. Static and dynamic analysis is the most commonly used approaches in Android malware detection BID2. In static analysis, android permissions are collected by unpacking or disassembling the app. In dynamic analysis, the run-time execution characteristics such as system calls, network connections, power consumption, user interactions and memory utilization. Mostly, commercial systems use combination of both the static and dynamic analysis. In Android devices, static analysis is preferred due to the following advantageous such as less computational cost, low resource utilization, light-weight and less time consuming. However, dynamic analysis has the capability to detect the metamorphic and polymorphic malwares. In BID3 evaluated the performance of traditional machine learning classifiers for android malware detection with using the permission, API calls and combination of both the API calls and permission as features. These 3 different feature sets were collected from the 2510 APK files. All traditional machine learning classifiers performance is good with combination of API calls and permission feature set in comparison to the API calls as well as permission. BID4 proposed MalDozer that use sequences of API calls with deep learning to detect Android malware and classify them to their corresponding family. The system has performed well in both private and public data sets, Malgenome, Drebin. Recently, the privacy and security for cloud computing is briefly discussed by BID5. The discussed various 28 cloud security issues and categorized those issues into five major categories. BID6 proposed machine learning based anomaly detection that acts on different layers e.g. the network, the service, or the workflow layers. BID7 discussed the issues in creating the intrusion detection for the cloud infrastructure. Also, how rule based and machine learning based system can be combined as hybrid system is shown. BID8 discussed the security problems in cloud and proposed incident detection system. They showed how incident detection system can perform well in comparison to the intrusion detection. In BID9 did comparative study of six different traditional machine learning classifiers in identifying the financial fraud. In BID10 discussed the applicability of data mining approaches for financial fraud detection. Deep learning is a sub model of machine learning technique largely used by researchers in recent days. This has been applied for various cyber security use cases BID11, BID12, BID13, BID14, BID15, BID16, BID17, BID18. Following, this paper proposes a unique DNN architecture which works efficiently on various cyber security use cases. The purpose of this section is to discuss the concepts of deep neural networks (DNNs) architecture concisely and promising techniques behind to train DNNs. Artificial neural networks (ANNs) represent a directed graph in which a set of artificial neuron generally called as units in mathematical model that are connected together with edges. This influenced by the characteristics of biological neural networks, where nodes represent biological neurons and edges represent synapses. A feed forward network is a type of ANNs. A feed forward network (FFN) consists of a set of units that are connected together with edges in a single direction without formation of a cycle. They are simple and most commonly used algorithm. Multi-layer perceptron (MLP) is a subset of FFN that consist of 3 or more layers with a number of artificial neurons, termed as units. The 3 layers are input layer, a hidden layer and output layer. There is a possibility to increase the number of hidden layers when the data is complex in nature. So, the number of hidden layer is parameterized and relies on the complexity of the data. These units together form an acyclic graph that passes information or signals in forward direction from layer to layer without the dependence of past input. MLP can be written as O: R p × R q where p and q are the size of the input vector x = x 1, x 2, · · ·, x p−1, x p and output vector O(x) respectively. The computation of each hidden layer Hl i can be mathematically formulated as follows. DISPLAYFORM0 DISPLAYFORM1 DISPLAYFORM2 DISPLAYFORM3 When then network consist of l hidden layers then the combined representation of them can be generally defined as, Rectified linear units (ReLU) have been turned out to be more proficient and are capable of accelerating the entire training process altogether BID19. Selecting ReLU is a more efficient way when considering the time cost of training the vast amount of data. The reason being that not only does it substantially speeds up the training process but also possesses some advantages when comparing to the traditional activation function including logistic sigmoid function and hyperbolic tangent function BID20. We refer to neurons with this nonlinearity following BID21. DISPLAYFORM4 We consider TensorFlow BID22 in conjunction with Keras BID23 as software framework. To increase the speed of gradient descent computations of deep learning architectures, we use with GPU enabled TensorFlow in single NVidia GK110BGL Tesla k40. All deep learning architectures are trained using the back propagation through time (BPTT) technique. Task 1 (Android Malware Classification): This data set includes 37,107 unique API information from 61,730 APK files BID24. These APK (application package) files were collected from the Opera Mobile Store over the period of January to September of 2014. When a user runs an application, a set of APIs will be called. Each API is related to a particular permission. The execution of the API may solely achieve success within the case that the permission is granted by the user. These permissions are grouped into Normal, Dangerous, Signature and Signature Or System in Android. These permissions are explicitly mentioned in the AndroidManifest.xml file of APK by application developers. Task 2 (Incident Detection): This dataset contains operational log file that was captured from Unified Threat Management (UTM) of UniteCloud BID25. UniteCloud uses resilient private cloud infrastructure to supply e-learning and e-research services for tertiary students and staffs in New Zealand. Unified Threat Management is a rule based real-time running system for UniteCloud server. Each sample of a log file contains nine features. These features are operational measurements of 9 different sensors in UTM system. Each sample is labeled based on the knowledge related to the incident status of the log samples. Task 3 (Fraud Detection): This dataset is anonymised data that was unified using the highly correlated rule based uniformly distributed synthetic data (HCRUD) approach by considering similar distribution of features BID26. The detailed statistics of Task 1, Task 2 and Task 3 data sets are reported in TAB0 In order to find an optimal learning rate, we run two trails of experiment till 500 epochs with learning rate varying in the range [0.01-0.5]. The highest 10-fold cross validation accuracy was obtained by using the learning rate of 0.1. There was a sudden decrease in accuracy at learning rate 0.2 and finally attained highest accuracy at learning rates of 0.35, 0.45 and 0.45 in comparison to learning rate 0.1. This accuracy may have been enhanced by running the experiments till 1000 epochs. As more complex architectures we have experimented with, showed less performance within 500 epochs, we decided to use 0.1 as learning rate for the rest of the experiments after considering the factors of training time and computational cost. The following network topologies are used in order to find an optimum network structure for our input data. 1) DNN 1 layer 2) DNN 2 layer 3) DNN 3 layer 4) DNN 4 layer 5) DNN 5 layer For all the above network topologies, we run 2 trails of experiments. Each trail of experiment was run till 500 epochs. It was observed that most of the deep learning architectures learn the normal category patterns of input data within 600 epochs. The number of epochs required to learn the malicious category data usually varies. The complex architecture networks required large number of iterations in order to reach the best accuracy. Finally, we obtained the best performed network topology for each use case. For Task 2 and Task 3, 4 layer DNNs network performed well. For Task 1, the performance of 5 layer DNNs network is good in comparison to the 4 layer DNNs. We decided to use 5 layer DNNs network for the rest of the experiments. 10-fold cross validation accuracy of each DNNs network topology for all use cases is shown in TAB0. An intuitive overview of proposed DNNs architecture, Deep-Net for all use cases is shown in FIG0 This contains an input layer, 5 hidden layer and output layer. An input layer contains 4896 neurons for Task 1, 9 neurons for Task 2 and 12 neurons for Task 3. An output layer contains 2 neurons for Task 1, 3 neurons for Task 2 and 2 neurons for Task 3. The details about the structure and configuration details of proposed DNNs architecture is shown in TAB0. The units in input to hidden layer and hidden to output layer are fully connected. DNNs network is trained using the backpropogation mechanism BID1. The proposed deep neural network is composed of fully-connected layers, batch normalization layers and dropout layers. Fully-connected layers: The units in this layer have connection to every other unit in the succeeding layer. Thats why this layer is called as fullyconnected layer. Generally, these fully-connected layers map the data into high dimension. The more the dimensions the data has the more accurate the data will be in determining the accurate output. It uses ReLU as non-linear activation function. Batch Normalization and Regularization: To obviate over fitting and speed up DNNs model training, Dropout (0.01) BID27 and Batch Normalization BID28 was used in between fully-connected layers. A dropout removes neurons with their connections randomly. In our alternative architectures for Task 1, the deep networks could easily overfit the training data without regularization even when trained on large number samples. Classification: For classification, the final fully connected layer follows sigmoid activation function for Task 1 and Task 2, sof tmax for Task 3. The fully connected layer absorb the non-linear kernel and sigmoid layer output 0 (benign) and 1 (malicious), sof tmax provides the probability score for each class. The prediction loss for Task 1 and Task 2 is estimated using binary cross entropy DISPLAYFORM0 where pd is a vector of predicted probability for all samples in testing data set, ed is a vector of expected class label, values are either 0 or 1.The prediction loss for Task 3 is estimated using categorical-cross entropy DISPLAYFORM1 where ed is true probability distribution, pd is predicted probability distribution. We have used sgd as an optimizer to minimize the loss of binary-cross entropy and categorical-cross entropy. We evaluate proposed DNNs model against classical machine learning classifier, on three different cyber security use cases. The first use case is identifying Android malware based on API information, the second use case is incident detection over unified threat management (UTM) operation on UniteCloud and the third use case is fraud detection in financial transactions. During training, we pass matrix of shape 30897*4896 for Task 1, 70000*9 for Task 2 and 70000*9 for Task 3 to the input layer of DNNs. These inputs are passed to more than one hidden layer (specifically 5) and output layer contains 1 neuron for Task 1 and Task 2, 3 neurons for Task TAB0 XGBoost is short for Extreme Gradient Boosting, where the term Gradient Boosting is proposed in the paper Greedy Function Approximation BID29. XGBoost is based on this original model. XGBoost is used for the given supervised learning problems (Task1, Task2 and Task3), where we use the training data (with multiple features) to predict a target variable. Here "multi:softmax" is used to perform the classification. After the observation and experiment, "max depth" of the tree set it as 20. 10 fold cross validation is performed to observe the training accuracy. Except Task 1, data are loaded as it is using Pandas 1. The "NaN" values are replaced with 0. In Task 1 the data is represented as a term -document matrix, where the vocabulary built using the API indication numbers in train and test. The scikit-learn BID11 count vectorizer is used to develop the termdocument matrix. On the successive representation, the data are fed to the XG Booster for prediction. The winner of CDMC 2017 tasks has acheived 0.9405, 0.9998 and 0.9824 on Task 1, Task 2 and Task 3 respectively using Random Forest classifier with Python scikit-learn BID11. The proposed method has performed well on Task 2 in comparision to the winner of CDMC 2017 and the accuracy obtained by DNNs varies -0.05%, -0.01% from the winner of CDMC 2017. The reported of DNNs can be further enhanced by simply adding hidden layers to the existing architecture that we are incompetent to try. Moreover, the proposed method can implicitly obtain the best features itself. This paper has evaluated the performance of deep neural networks (DNNs) for cyber security uses cases: Android malware classification, incident detection and fraud detection. Additionally, other classical machine learning classifier is used. In all cases, the performance of DNNs is good in comparison to the classical machine learning classifier. Moreover, the same architecture is able to perform better than the other classical machine learning classifier in all use cases. The reported of DNNs can be further improved by promoting training or stacking a few more layer to the existing architectures. This will be remained as one of the direction towards the future work.

Represent the short summary for finding the corresponding scientific article (long)

Deep-Net: Deep Neural Network for Cyber Security Use Cases

Information about whether a neural network's output will be correct or incorrect is somewhat present in the outputs of the network's intermediate layers.


Question: Represent the scientific article to find a short TLDR

Question: In this paper, we present an approach to learn recomposable motor primitives across large-scale and diverse manipulation demonstrations. Current approaches to decomposing demonstrations into primitives often assume manually defined primitives and bypass the difficulty of discovering these primitives. On the other hand, approaches in primitive discovery put restrictive assumptions on the complexity of a primitive, which limit applicability to narrow tasks. Our approach attempts to circumvent these challenges by jointly learning both the underlying motor primitives and recomposing these primitives to form the original demonstration. Through constraints on both the parsimony of primitive decomposition and the simplicity of a given primitive, we are able to learn a diverse set of motor primitives, as well as a coherent latent representation for these primitives. We demonstrate, both qualitatively and quantitatively, that our learned primitives capture semantically meaningful aspects of a demonstration. This allows us to compose these primitives in a hierarchical reinforcement learning setup to efficiently solve robotic manipulation tasks like reaching and pushing. We have seen impressive progress over the recent years in learning based approaches to perform a plethora of manipulation tasks;; ). However, these systems are typically task-centric savants -able to only execute a single task that they were trained for. This is because these systems, whether leveraging demonstrations or environmental rewards, attempt to learn each task tabula rasa, where low to high level motor behaviours, are all acquired from scratch in context of the specified task. In contrast, we humans are adept at a variety of basic manipulation skills e.g. picking, pushing, grasping etc., and can effortlessly perform these diverse tasks via a unified manipulation system. Sample motor programs that emerge by discovering the space of motor programs from a diverse set of robot demonstration data in an unsupervised manner. These motor programs facilitate understanding the commonalities across various demonstrations, and accelerate learning for downstream tasks. How can we step-away from the paradigm of learning task-centric savants, and move towards building similar unified manipulation systems? We can begin by not treating these tasks independently, but via instead exploiting the commonalities across them. One such commonality relates to the primitive actions executed to accomplish the tasks -while the high-level semantics of tasks may differ significantly, the low and mid-level motor programs across them are often shared e.g. to either pick or push an object, one must move the hand towards it. This concept of motor programs can be traced back to the work of Lashley, who noted that human motor movements consist of'orderly sequences' that are not simply sequences of stimulus-response patterns. The term'motor programs' is however better attributed to as being representative of'muscle commands that execute a movement sequence uninfluenced by peripheral feedback', though later works shifted the focus from muscle commands to the movement itself, while allowing for some feedback . More directly relevant to our motivation is Schmidt's notion of'generalized' motor programs that can allow abstracting a class of movement patterns instead of a singular one. In this work, we present an approach to discover the shared space of (generalized) motor programs underlying a variety of tasks, and show that elements from this space can be composed to accomplish diverse tasks. Not only does this allow understanding the commonalities and shared structure across diverse skills, the discovered space of motor programs can provide a high-level abstraction using which new skills can be acquired quickly by simply learning the set of desired motor programs to compose. We are not the first to advocate the use of such mid-level primitives for efficient learning or generalization, and there have been several reincarnations of this idea over the decades, from'operators' in the classical STRIPS algorithm , to'options' or'primitives' in modern usage. These previous approaches however assume a set of manually defined/programmed primitives and therefore bypass the difficulty of discovering them. While some attempts have been made to simultaneously learn the desired skill and the underlying primitives, learning both from scratch is difficult, and are therefore restricted to narrow tasks. Towards overcoming this difficulty, we observe that instead of learning the primitives from scratch in the context of a specific task, we can instead discover them using demonstrations of a diverse set of tasks. Concretely, by leveraging demonstrations for different skills e.g. pouring, grasping, opening etc., we discover the motor programs (or movement primitives) that occur across these. We present an approach to discover movement primitives from a set of unstructured robot demonstration i.e. demonstrations without additional parsing or segmentation labels available. This is a challenging task as each demonstration is composed of a varying number of unknown primitives, and therefore the process of learning entails both, learning the space of primitives as well as understanding the available demonstrations in context of these. Our approach is based on the insight that an abstraction of a demonstrations into a sequence of motor programs or primitives, each of which correspond to an implied movement sequence, and must yield back the demonstration when the inferred primitives are'recomposed'. We build on this and formulate an unsupervised approach to jointly learn the space of movement primitives, as well as a parsing of the available demonstrations into a high-level sequence of these primitives. We demonstrate that our method allows us to learn a primitive space that captures the shared motions required across diverse skills, and that these motor programs can be adapted and composed to further perform specific tasks. Furthermore, we show that these motor programs are semantically meaningful, and can be recombined to solved robotic tasks using reinforcement learning. Specifically, solving reaching and pushing tasks with reinforcement learning over the space of primitives achieves 2 orders of magnitude faster training than reinforcement learning in the low-level control space. Our work is broadly related to several different lines of work which either learn task policies from demonstrations, or leverage known primitives for various applications, or learn primitives in context of known segments. While we discuss these relations in more detail below, we note that previous primitive based approaches either require: a) a known and fixed primitive space, or b) annotated segments each corresponding to a primitive. In contrast, we learn the space of primitives without requiring this segmentation annotation, and would like to emphasize that ours is the first work to do so for a diverse set of demonstrations spanning multiple tasks. Learning from Demonstration: The field of learning from demonstrations (LfD) has sought to learn to perform tasks from a set of demonstrated behaviors. A number of techniques exist to do so, including cloning the demonstrated behavior , fitting a parametric model to the demonstrations , or first segmenting the demonstrations and fitting a model to each of the ant segments (; ; ;). We point the reader to for a comprehensive overview of the field. Rather than directly learn to perform tasks, we use demonstrations to learn a diverse set of composable and reusable primitives, or motor-programs that may be used to perform a variety of downstream tasks. Learning and Sequencing Motion Primitives: Several works learn motion primitives from demonstrations using predetermined representations of skills such as Dynamic Movement Primitives (DMPs) . A few other works have approached the problem from an optimization perspective . Given these primitives, a question that arises is how to then sequence learned skills to perform downstream tasks. Several works have attempted to answer this question - builds a layered approach to adapt, select, and sequence DMPs.; segments demonstrations into sequences of skills, and also merge these skills into skill-trees. However, the predetermined representations of primitives adopted in these works can prove restrictive. In particular, it prevents learning arbitrarily expressive motions and adapting these motions to a generic downstream task. We seek to move away from these fixed representations of primitives, and instead learn representations of primitives along with the primitives themselves. The family of latent variable models (LVMs) provides us with fitting machinery to do so. Indeed, the success of LVMs in learning representations in deep learning has inspired several recent works (; ; ;) to learn latent representations of trajectories. SeCTAR builds a latent variable conditioned policy and model that are constrained to be consistent with one another, and uses the learned policies and model for hierarchical reinforcement learning. The CompILE framework seeks to learn variable length trajectory segments from demonstrations instead, and uses latent variables to represent these trajectory segments, but is evaluated in relatively low-dimensional domains. We adopt a similar perspective to these works and learn continuous latent variable representations (or abstractions) of trajectory segments. Hierarchical RL and the Options Framework: The related field of hierarchical reinforcement learning (HRL) learns a layering of policies that each abstract away details of control of the policies of a lower level. The Options framework also learns similar temporal abstractions over sequences of atomic actions. While promising, its application has traditionally been restricted to simple domains due to the difficulty of jointly learning internal option policies along with a policy over those options. Recent works have managed to do so with only a reward function as feedback. The Option-Critic framework employs a policy-gradient formulation of options to do so, while the Option-Gradient learns options in an off-policy manner. In contrast with most prior work in the options framework, (; learn options from a set of demonstrations, rather than in the RL setting. In similar spirit to these works, we too seek to learn abstractions from a given set of demonstrations, however unlike DDO, DDCO, and CompILE , we can learn primitives beyond a discrete set of options in a relatively high dimensional domain. Hierarchical representations of demonstrations: The idea of hierarchical task representations has permeated into LfD as well. In contrast to reasoning about demonstrations in a flat manner, one may also infer the hierarchical structure of tasks performed in demonstrations. A few recent works have striven to do so, by representing these tasks as programs , or as task graphs . and address generalizing to new instances of manipulation tasks in the low-shot regime by abstracting away low-level controls. The idea of policy sketches, i.e. a sketch of the sub-tasks to be accomplished in a particular task, has become popular . learn modular policies in the RL setting provided with such policy sketches. provides a modular LfD framework based on this idea of policy sketches. While all of these works address learning policies at various levels from demonstrations, unlike our approach, they each assume access to heavy supervision over demonstrations to do so. We seek to discover the space of motor programs directly from unstructured demonstrations in an unsupervised manner, and show that these can help understanding similarities across tasks as well as quickly quickly adapting to and solving new tasks. Building on ideas of and , we define a motor program M as a movement pattern that may be executed in and of itself, without access to sensory feedback. Concretely, a'movement sequence' or a motor program M is a sequence of robot joint configurations. Our goal is to learn the space of such movement patterns that Figure 2: An overview of our approach. Our abstraction network takes in an observed demonstration τ obs and predict a sequence of latent variables {z}. These {z} are each decoded into their corresponding motor programs via the motor program network. We finally recompose these motor programs into the recomposed trajectory. are present across a diverse set of tasks. We do so via learning a'motor program network' that maps elements z ∈ R n to corresponding movement sequences i.e. M: z −→ M. Given a set of N unlabelled demonstrations {τ i} i=1,2,...,N that consist of sequences of robot states {s 1, s 2, ..., s T} ∈ S, our aim is to learn the shared space of motor programs via the motor program network M. However, as each demonstration τ i is unannotated, we do not apriori know what subsequences of the demonstration correspond to a distinct motor program. Therefore, to learn motor programs from these demonstrations, we need to simultaneously learn to understand the demonstrated trajectories in terms of composition of motor programs. Thus in addition to learning the network M, we also learn a mapping A from each demonstrated trajectory τ i to the underlying sequence of motor programs {M 1, M 2, ..., M K} (and associated latent variables {z 1, z 2, ..., z K}) executed during the span of the trajectory. We call this mapping A: as it abstracts away the details of the trajectory into the set of motor programs (i.e., abstractions). Note that both the abstraction and motor program networks are learned using only a set of demonstrations from across diverse tasks. Our central insight is that we can jointly learn the space of motor programs and the abstraction of the demonstration by enforcing that the implied'recomposition' is faithful to the original demonstration. Concretely, an abstraction of a demonstration into a sequence of motor programs, each of which corresponds to an implied motion sequence, must yield back the original demonstration when the inferred motor programs are decoded and'recomposed'. We operationalize this insight to jointly train the motor program and abstraction networks from demonstrations. Learning Overview and Objective: Our approach is outlined in Fig 2, where given an input demonstration trajectory τ obs, we use the abstraction network to predict a sequence of (a variable number of) latent codes {z k}. These are each decoded into corresponding sub-trajectories via the learned motor program network M. Given the decoding of the predicted motor programs, we can recompose these sub-trajectories to obtain a recomposed demonstration trajectory τ rec, and penalize the discrepancy between the observed and the recomposed demonstrations. Denoting by ⊕ the concatenation operator, our loss for a given demonstration τ obs is therefore characterized as: As the trajectories τ obs, τ rec are possibly of different lengths, we use a pairwise matching cost between the two trajectories, where the optimal alignment is computed via dynamic time warping . This provides us with a more robust cost measure that handles different prediction lengths, is invariant to minor velocity perturbations, and enables the model to discard regions of inactivity in the demonstrations. Given two trajectories τ a ≡ (, and a distance metric δ over the state space, the discrepancy measure between trajectories can be defined as the matching cost for the optimal matching path P among all possible valid matching paths P (i.e. paths satisfying monotonicity, continuity, and boundary conditions ): As the recomposed trajectory comprises of distinct primitives, each of which implies a sequence of states, we sometimes observe discontinuities i.e. large state changes between the boundaries of these primitives. To prevent this, we additionally incorporate a smoothness loss L sm (τ rec) that penalizes the state change across consecutive time-steps if they are larger than a certain margin. Our overall objective, comprising of the reconstruction objective and the smoothness prior, can allow us to jointly learn the space of motor programs and the abstraction of trajectories in an unsupervised manner. Network Architecture and Implementation Details. We parameterize our motor program network M and our abstraction network A as neural networks. In particular, the motor program network is a 4 layer LSTM that takes a single 64 dimensional latent variable z as input, and predicts a sequence of 16 dimensional states. For our abstraction network, we adopt the Transformer architecture to take in a varying length 16 dimensional continuous joint angle trajectory τ as input, and predict a variable number of latent variables {z}, that correspond to the sequence of motor programs {M} executed during trajectory τ. We find the transformer architecture to be superior to LSTMs for processing long trajectories, due to its capacity to attend to parts of the trajectory as required. Our abstraction network A predicts a varying number of primitives by additionally predicting a'continuation probability' p k after each motor program variable z k. We then predict an additional primitive only if the sampled discrete variable from p k is 1, and therefore also need to learn the prediction of these probabilities. While the loss function above can directly yield gradients to the predicted motor program encoding z k via M, we use gradients using (with ∆(τ obs, τ rec) + L sm (τ rec) as negative reward) to learn prediction of p k. While the objective described so far can in principle allow us to jointly learn the space of motor programs and understand the demonstration trajectories as a composition of these, there are additional properties we would wish to enforce to the bias the learning towards more desirable solutions. As an example, our framework presented so far can allow a solution where each demonstration is a motor program by itself i.e. the abstraction network can learn to map each demonstration to a unique z, and the primitive decoder can then decode this back. However, this is not a suitable solution as the learned motor programs are not'simple'. On the other extreme, a very simple notion of a motor program is one that models each transition independently. However, this is again undesirable as this does not'abstract' away the details of control or represent the demonstration as a smaller number of motor programs. Therefore, in addition to enforcing that the learned motor programs recompose the demonstrations, we also need to enforce simplicity and parsimony of these motor programs. We incorporate these additional biases by adding priors in the objective or model space. To encourage the abstraction model to learn parsimonious abstractions of the input demonstrations, we penalize the number of motor primitives used to recompose the trajectory, by adding a small constant to the negative reward used to train the continuation probability p k if the corresponding sample yielded an additional primitive. To enforce simplicity of motor primitives, we observe that the trajectories yielded by a classical planner (in our case, RRT-Connect) e.g. to go from an initial to final state are'simple' and we therefore initialize the motor primitive network using the decoder of a pretrained autoencoder on random planner trajectories for (start, goal) state pairs. Note that this notion of'plannability' as'simplicity' is merely one plausible alternative, and alternate ones can be explored e.g.'linearity' or'predictability of motion' . We would like to ascertain how well our approach is capable of achieving our objective of successfully discovering and learning a good representation of the space of motor programs. Further, we seek to verify whether despite being learned in an unsupervised manner without semantic grounding, the learned primitive space is semantically meaningful. We would also like to evaluate how well they can be used to solve downstream RL tasks. We first provide a description of the data we wish to learn primitives from, followed by describing our quantitative and qualitative experiments towards verifying these three axes. Dataset: We use the MIME dataset to train and evaluate our model. The dataset consists of over 8000 kinesthetic demonstrations of 20 tasks (such as pouring, pushing, bottle opening, stacking objects, etc.) collected on a real-world Baxter Robot. While the dataset has head and hand-mounted RGBD data, we use the Baxter joint angle trajectories to train our model. We consider a 16 dimensional space as our input and prediction space, consisting of 7 joints for each of the 2 arms, along with a scalar value for each gripper (we ignore torso and head joints). We emphasize this is a higher dimensional domain than most other related works consider. The gripper values are re-scaled to a 0 − 1 range, while the joint angles are unnormalized. We temporally down-sample all joint angle data by a constant factor of 20 from the original data frequency of 100 Hz. We randomly sample a train set of 5900 demonstrations from all 20 tasks, with a validation set of 1600 trajectories, and a held-out test set of 850 trajectories. To help evaluate the learned motor programs, we manually annotate a set of 60 test trajectories (3 trajectories from each task) with temporal segmentation annotations, as well as semantic labels of 10 primitives (such as reaching, twisting, pushing, etc.) that occur in these 60 trajectories. Note that these labels are purely for evaluation, our model does not have access to these annotations during training. We would first like to evaluate the quality of the learned abstractions in and of themselves, i.e. Is our approach able to discover the space of motor programs, and learn a good representation of this space? We qualitatively answer this question by visualizing the latent representation space of motor primitives learned by our model. We first randomly sample a set of 500 trajectories unseen during training, then pass these trajectories through our model, and retrieve the predicted latent variables {z} for each of these trajectories and their corresponding movement sequences {τ}. We then embed the latent variables in a 2-dimensional space using T-SNE (van der), and visualize the corresponding movement sequences at their corresponding position in this 2-D embedded space, as in Fig. 3. We provide a GIF version of Fig. 3 (and other visualizations) at https://sites.google.com/view/discovering-motor-programs/home. We observe that clusters of movement sequences emerge in this embedded space based on the relative motions executed during the course of these trajectory segments (similar latent variables correspond to similar movement sequences and vice versa). While these clusters are not explicitly semantically labelled by our approach, the motions in these clusters correlate highly with traditional notions of skills in robot manipulation, i.e. reaching motions (top and top-left clusters), returning motions (bottom cluster), bi-manual motions (visible to the bottom right of the left most cluster and the bottom of the right most cluster), etc. We visualize a few such primitives (reaching, twisting, grasping, bi-manual pulling, etc.) among these to the right of Fig. 3. This shows our model learns smooth mappings M and A, and is capable of discovering such primitives in an unsupervised manner without explicit temporal segmentation or semantic labels, which we believe is an encouraging . Interestingly, the model learns abstractions that pick up on the trend of the motion, rather than distinguishing between whether the left or right hand is used for the motion. This is particularly notable in the case of reaching and returning motions, where both left and right-handed reaching and returning motions appear alongside each other in their respective clusters in the embedded space. We would ideally like the learned primitives from our model to be useful on a real Baxter robot platform, and be suitably smooth, feasible, and correspond to the motions executed in simulation (i.e. be largely unaffected by the noise of execution on a real robot). To verify whether our model is indeed able to learn such primitives, we execute a small set of learned primitives on a real Baxter robot, by feeding in the trajectory predicted by the model into a simple position controller. We visualize the of this execution in Fig. 4, (see project webpage for videos). Despite not explicitly optimizing for feasibility or transfer to a real robot, the use of real-world Baxter data to train our model heavily biases the model towards primitives that are inherently feasible, relatively smooth and can be executed on a real robot without any subsequent modifications. As the'recomposed' trajectory can be aligned to the original demonstration via sequence alignment, our predicted abstractions induce a partitioning of the demonstrated trajectory (corresponding to the aligned boundaries of the predicted primitives). We test whether the predicted abstraction and induced partitions are consistent across different demonstrations from the same task. To this end, we select 3 instances of the "Drop Object" task from the annotated test set, and retrieve the induced segmentations of the demonstration. We then visualize these segmentations and motor programs predicted for each of these 3 demonstrations as depicted in Fig. 5, along with the ground truth semantic labels of primitives for each of the demonstrations. The alignment between a recomposed trajectory and the original demonstration also allows us to transfer semantic annotations from a demonstration to the predicted primitives, by simply copying labels from the demonstration to their aligned timepoints in the primitives. Therefore using our small set of annotated demonstrations, we can construct a small library of semantically annotated primitives. Given a novel, unseen demonstration, we can compute its predicted primitives and assign each a semantic label by copying the label of the nearest primitive from the library. This allows us to transfer semantic segmentations from our small annotated test set to unseen demonstrations. Our model's transfer of semantic segmentation achieves label accuracy on the set of 30 held out trajectories (across all 20 tasks) of 58%, while a supervised LSTM baseline that predicts semantic labels from trajectories (trained on 30 annotated test trajectories) achieves 54% accuracy. The consistency of our abstractions coupled with the ability to transfer semantic segmentations across demonstrations shows our model is capable of understanding commonalities across demonstrations, and is able to reason about various demonstrations in terms of motor programs shared across them, despite being trained in an unsupervised manner. Each row represents a different instance of a "Drop Objects" task from the MIME Dataset, while each column represents a time-step in the demonstration. White frames represent predicted segmentation points, while colored boxes represent ground truth semantic annotations. Red boxes are reaching primitives, blue boxes are grasping, orange boxes are placing, and green boxes are returning primitives. We see our model predicts 4 motor programs -reaching, grasping, placing the object a small distance away, and returning. This is consistent with the true semantic annotations of these demonstrations, and the overall sequence of primitives expected of the "Drop Box" task. (a) Sparse Baxter Reacher Task (b) Sparse Baxter Pushing Task Figure 6: RL training curves with and without motor programs. Solid lines denote mean success rate, while shaded region denotes ±1 standard deviation across 10 random seeds. One of our primary motivations for learning motor programs is that they can be composed together to solve downstream robotic tasks. To evaluate whether the motor programs learned by our model are indeed useful for such downstream tasks, we adopt a hierarchical reinforcement learning setup (as described in detail in the supplementary). For a given task, we train a policy to predict the sequence of motor programs to execute. Given the predicted latent representations, each motor program is decoded into its corresponding motion sequence using the previously learned motor program network. We retrieve a sequence of desired joint velocities from this motion sequence and use a joint velocity controller to execute these "low-level" actions on the robot. The motor program network and the joint velocity controller together serve as an "abstraction" of the low-level control that is executed on the robot. Hence, the policy must learn to predict motor programs that correspond to motion sequences useful for solving the task at hand. As demonstrated in Fig. 6, training a policy using motor programs is several orders of magnitude more efficient than training with direct low-level actions. For the sparse reaching task, the motor program policy learns within 50 motor program queries, 2 orders of magnitude speedup in low-level control time-steps. For the sparse pushing task, the motor program policy learns within 1000 motor program queries, or a 2X speedup with respect to low-level control time-steps. We note that executing a motor program corresponds to 50 low level control steps (see appendix for details). However, as these motor programs are executed without environment feedback, the improvement in efficiency in terms of environment interactions is all the more significant. We have presented an unsupervised approach to discover motor programs from a set of unstructured robot demonstrations. Through the insight that learned motor programs should recompose into the original demonstration while being simplistic, we discover a coherent and diverse latent space of primitives on the MIME dataset. We also observed that the learned primitives were semantically meaningful, and useful for efficiently learning downstream tasks in simulation. We hope that the contributions from our work enable learning and executing primitives in a plethora of real-world robotic tasks. It would also be interesting to leverage the learned motor programs in context of continual learning, to investigate how the discovered space can be adapted and expanded in context of novel robotic tasks. We provide additional visualizations of primitives being executed on the real robot below. As mentioned in the main paper, dynamic GIFs of these visualizations may be found at https://sites.google.com/view/discovering-motor-programs/home. Figure 7: Depiction of execution of additional learned primitives on real world Baxter robot. As in Fig. 4, each row is a single primitive, while columns show progress of the primitive over time. Row 1 depicts a left handed returning primitive, row 2 depicts a right handed pushing primitive, row 3 depicts a left handed pushing primitive (in a different configuration to the left handed one), and finally row 4 depicts a right handed twisting primitive. For our hierarchical reinforcement learning experiments, we perform policy learning on two sparse reward tasks on a simulated Baxter Robot: (a) Reaching, and (b) Pushing. For the reaching task, the robot's end-effector needs to reach a specific location in space, while for the pushing task, the robot needs to push a block on the table to a specific location. For the tasks, the policy gets a reward of 1 if it is within 0.05m of the goal and reward of 0 otherwise. We train both our motor program policy and the baseline control policy using Proximal Policy Optimization . Note that the motor program policy outputs the latent representation z. Each z expands into a 10 length trajectory according the motor program network. To reach each of these trajectory states, a PD velocity controller is used for 5 time-steps. The baseline control policy directly outputs the velocity control action.

We learn a space of motor primitives from unannotated robot demonstrations, and show these primitives are semantically meaningful and can be composed for new robot tasks.

We propose a novel Intrinsically Motivated Goal Exploration architecture with unsupervised learning of goal space representations, and evaluate how various implementations enable the discovery of a diversity of policies.


Question: Represent the scientific article to find a short TLDR

Question: Using modern deep learning models to make predictions on time series data from wearable sensors generally requires large amounts of labeled data. However, labeling these large datasets can be both cumbersome and costly. In this paper, we apply weak supervision to time series data, and programmatically label a dataset from sensors worn by patients with Parkinson's. We then built a LSTM model that predicts when these patients exhibit clinically relevant freezing behavior (inability to make effective forward stepping). We show that when our model is trained using patient-specific data (prior sensor sessions), we come within 9% AUROC of a model trained using hand-labeled data and when we assume no prior observations of subjects, our weakly supervised model matched performance with hand-labeled data. These demonstrate that weak supervision may help reduce the need to painstakingly hand label time series training data. Time series data generated by wearable sensors are an increasingly common source of biomedical data. With their ability to monitor events in non-laboratory conditions, sensors offer new insights into human health across a diverse range of applications, including continuous glucose monitoring BID1, atrial fibrillation detection BID11, fall detection BID2, and general human movement monitoring BID6.Supervised machine learning with sensor time series data can help automate many of these monitoring tasks and enable medical professionals make more informed decisions. However, developing these supervised models is challenging due to the cost and difficultly in obtaining labeled training data, especially in settings with considerable inter-subject variability, as is common in human movement research BID5. Traditionally, medical professionals must hand label events observed in controlled laboratory settings. When the events of interest are rare this process is time consuming, expensive, and does not scale to the sizes needed to train robust machine learning models. Thus there is a need to efficiently label the large amounts of data that machine learning algorithms require for time series tasks. In this work, we explore weakly supervised BID10 ) models for time series classification. Instead of using manually labeled training data, weak supervision encodes domain insights into the form of heuristic labeling functions, which are used to create large, probabilistically labeled training sets. This method is especially useful for time series classification, where the sheer number of data points makes manual labeling difficult. As a motivating test case, we focus on training a deep learning model to classify freezing behaviors in people with Parkinson's disease. We hypothesize that by encoding biomechanical knowledge about human movement and Parkinson's BID5 into our weakly supervised model, we can reduce the need for large amounts of hand labeled data and achieve similar performance to fully supervised models for classifying freezing behavior. We focus on two typical clinical use cases when making predictions for a patient: where we have no prior observations of the patient, and where we have at least one observation of the patient. In weak supervision, noisy training labels are programmatically generated for unlabeled data using several heuristic labeling functions which encode specific domain knowledge. These labeling functions are modeled as a generative process which allows us to denoise the labels by learning their correlation structure and accuracies BID10. These labeling functions are of the form λ: X → Y ∪ ∅ which take in a single candidate x ∈ X, and output a label y ∈ Y or ∅, if the function abstains. Using n labeling functions on m unlabeled data points, we create a label matrix L = (Y ∪ ∅) m×n. We then create a generative model from this label matrix and three factor types (labeling propensity, accuracy, and pairwise correlation) of labeling functions: DISPLAYFORM0 where C are the potential correlations. Next, we concatenate all these factors for a given data point x i and all labeling functions j = 1...n, ing in φ i (L, Y), and learn the parameters w ∈ R 2n+|C| to maximize the objective: DISPLAYFORM1 With this generative model, we can then generate probabilistic training labels,Ỹ = pŵ(Y |L) wherê w are the learned parameters in the label model. Using these probabilistic labels, we can train a discriminative model that we aim to generalize beyond the information encoded in the labeling functions. We do this by minimizing the expected loss with respect toŶ:θ DISPLAYFORM2 As we increase the amount of unlabeled data, we increase predictive performance BID9. We use a dataset that contains series of measurements from 36 trials from 9 patients that have Parkinson's Disease (PD) and exhibit freezing behavior. PD is a neurodegenerative disease marked by tremor, loss of balance, and other motor impairments, that affects over 10 million people worldwide. Freezing of gait (FOG) -a sudden and brief episode where an individual is unable to produce effective forward stepping BID3 BID4 -is one of the disabling problems caused by PD, and often leads to falls BID0.In this dataset, subjects walked in a laboratory setting that the investigators designed to elicit freezing events. Leg or shank angular velocity was measured during the forward walking task using wearable inertial measurement units (sampled at 128 Hz), which were positioned in a standardized manner for all subjects and tasks on the top of the feet, on both shanks, on the lumbar, and chest trunk regions. In this work, we focus on sensor streams from both shanks, though it is straightforward to include the other sensor streams (e.g. lumbar, feet, etc.). From each Turning and Barrier Course run, we extract left and right ankle gyroscope data in the z-direction from each trial (up to 4 trials per course run), along with the gold labels for these trials, which were manually recorded by a neurologist. We combine the data from all trials from all course runs, and segment the sensor data by gait cycle (of the right leg) which is computed analytically from the angular velocity of ankle sensor data. In this case, we define gait cycle as the time period between two successive peaks on an angular velocity versus time plot. We then define a single candidate to be x a×b ∈ X where a is the number of sensor streams and b is the sequence length. For our task, a = 2 since we use the left and right ankle sensor streams, and b is the sequence length for a single gait cycle (which slightly varies from cycle to cycle). To programatically label data, we use five labeling functions which draw on domain specific knowledge and empirical observations. Specifically, these labeling functions target features which can distinguish freezing and non-freezing events. For all labeling functions, we assign positive, negative, or abstain labels based on empirically measured threshold values from the validation set. For example, one heuristic we employ uses stride time arrhythmicity BID7 BID8, which we calculate as average coefficient of variation for the past 3 stride times of the left and right leg. For this function, we label a candidate as freezing if the arrhythmicity of that candidate is greater than 0.55, and not freezing if the arrhythmicity is less than 0.15. If arrhythmicity for a particular candidate is in between these two values, we abstain. Other labeling functions we use involve the swing angular range of the shank, and the amplitude and variance in shank angular velocity. Using these labeling functions, we build a generative label model and predict probabilistic labels y ∈ Y for each candidate x a×b ∈ X in the training set FIG0 ). See TAB0 for the individual performance of each labeling function. We then train a discriminative model on the probabilistic labels from the generative model that incorporates the labeling functions discussed in the last section. We use a single layer bi-directional LSTM and hidden state dimension 300 for our end model that takes in a multivariate sensor stream as input. Since we use time series data from only the left and right ankle sensors, our input is two dimensional. In order to provide longer temporal context, we pass in a windowed version of each candidate that includes the last three gait cycles and the next gait cycle FIG0 ). Since sequence length of a single gait cycle slightly varies, we then pad these sequences and truncate any sequences over a pre-defined maximum sequence length. To provide more contextual signal, we also add multiplicative attention to pool over the hidden states in the LSTM. We evaluate our weakly supervised model in two typical clinical settings, and compare performance with that of a fully supervised model. In the first setting, we split the data into training/validation/testing by trials/sessions. In this setting, both the validation and testing set have a single trial from each patient, and the training set has one or more trials from each patient. In the second setting, we split data by patient. In this case, the testing (and validation) set contains all the trials from a novel patient. We then cross-validate on each patient. These are summarized in TAB1.From the session splits setting, we note that our weakly supervised model comes within 10 points in F1 score and 8 points in AUROC of the fully supervised (hand-labeled) model. In the patient splits setting, our weakly supervised model matches the performance of the fully supervised model. In both supervision types, it is clear that our end to end system performs significantly better when it has seen 1 or more sessions of a particular patient before. In the patient splits setting, our system has difficulty generalizing to certain patients. For example, in TAB2 we see that the fully supervised model has trouble predicting freezing events for patients P1, P7, and P8 in particular. These difficulties are inherent to the problem -each patient exhibits different freezing behaviors, and some, such as P1, P2, and P8, have relatively rare freezing events. This highlights why, at least for this task, it is critical to have as many subjects as possible in the dataset -an objective that is far easier to meet if hand labels are not required. Our work demonstrates the potential of weak supervision on time series tasks. In both experiments, our weakly supervised models performed close to or match the fully supervised models. Further, the amount of data available for the weak supervision task was fairly small -with more unlabeled data, we expect to be able to improve performance BID9. These show that costly and time-intensive hand labeling may not be required to get the desired performance of a given classifier. In the future, we plan to add more and different types of sensor streams and modalities (e.g., video). We also plan to use labeling functions to better model the temporal correlation between individual segments of these streams, which can potentially improve our generative model and hence end to end performance.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We demonstrate the feasibility of a weakly supervised time series classification approach for wearable sensor data.

Represent the TLDR for finding the corresponding scientific article

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.


Question: Represent text!

Question: Learning semantic correspondence between the structured data (e.g., slot-value pairs) and associated texts is a core problem for many downstream NLP applications, e.g., data-to-text generation. Recent neural generation methods require to use large scale training data. However, the collected data-text pairs for training are usually loosely corresponded, where texts contain additional or contradicted information compare to its paired input. In this paper, we propose a local-to-global alignment (L2GA) framework to learn semantic correspondences from loosely related data-text pairs. First, a local alignment model based on multi-instance learning is applied to build the semantic correspondences within a data-text pair. Then, a global alignment model built on top of a memory guided conditional random field (CRF) layer is designed to exploit dependencies among alignments in the entire training corpus, where the memory is used to integrate the alignment clues provided by the local alignment model. Therefore, it is capable of inducing missing alignments for text spans that are not supported by its imperfect paired input. Experiments on recent restaurant dataset show that our proposed method can improve the alignment accuracy and as a by product, our method is also applicable to induce semantically equivalent training data-text pairs for neural generation models. Learning semantic correspondences between the structured data (e.g., slot-values pairs in a meaning representation (MR)) and associated description texts is one of core problem in NLP community , e.g., data-to-text generation produces texts based on the learned semantic correspondences. Recent data-to-text generation methods, especially neural-base methods which are data-hungry, adopt data-text pairs collected from web for training. Such collected corpus usually contain loosely corresponded data text pairs , where text spans contain information that are not supported by its imperfect structured input. Figure 1 depicts an example, where the slot-value pair Price=Cheap can be aligned to text span low price range while the text span restaurant doesn't supported by any slot-value pair in paired input MR. Most of previous work for learning semantic correspondences (; ; ;) focus on characterizing local interactions between every text span with a corresponded slots presented in its paired MR. Such methods cannot work directly on loosely corresponded data-text pairs, as setting is different. In this work, we make a step towards explicit semantic correspondences (i.e., alignments) in loosely corresponded data text pairs. Compared with traditional setting, which only attempts inducing alignments for every text span with a corresponded slot presented in its paired MR. We propose a Local-to-Global Alignment (L2GA) framework, where the local alignment model discovers the correspondences within a single data-text pair (e.g., low price range is aligned with the slot Price in Figure 1) and a global alignment model exploits dependencies among alignments presented in the entire data-text pairs and therefore, is able to induce missing attributes for text spans not supported in its noisy input data (e.g., restaurant is aligned with the slot EatType in Figure 1). Specially, our proposed L2GA is composed of two parts. The local alignment model is a neural method optimized via a multi-instance learning paradigm which automatically captures correspondences by maximizing the similarities between co-occurred slots and texts within a data-text pair. Our proposed global alignment model is a memory guided conditional random field (CRF) based sequence labeling framework. The CRF layer is able to learn dependencies among semantic labels over the entire corpus and therefore is suitable for inferring missing alignments of unsupported text spans. However, since there are no semantic labels provided for sequence labeling, we can only leverage limited supervision provided in a data-text pair. We start by generating pseudo labels using string matching heuristic between words and slots (e.g., Golden Palace is aligned with Name in Figure 1). The pseudo labels in large portion of unmatched text spans (e.g., low price and restaurant cannot be directly matched in Figure 1), we tackle this challenge by: a) changing the calculation of prediction probability in CRF layer, where we sum probabilities over possible label sequences for unmatched text spans to allow inference on unmatched words; b) incorporating alignment produced by the local alignment model as an additional memory to guide the CRF layer, therefore, the semantic correspondences captured by local alignment model can together work with the CRF layer to induce alignments locally and globally. We conduct experiments of our proposed method on a recent restaurant dataset, E2E challenge benchmark (a), show that our framework can improve the alignment accuracy with respect to previous methods. Moreover, our proposed method can explicitly detect unaligned errors presented in the original training corpus and provide semantically equivalent training data-text pairs for neural generation models. Experimental also show that our proposed method can improve content consistency for neural generation models. Here, we provide a brief description of learning alignments in loosely corresponded data-text pairs. Given a corpus with paired meaning representations (MR) and text descriptions {(R, X)} N i=1. The input MR R = (r 1, . . ., r M) is a set of slot-value pairs r j = (s j, v j), where each r j contains a slot s j (e.g., Price) and a value v j (e.g., Cheap). According to s j, the corpus has K unique slots in total, where K >= M. The corresponding description X = (x 1, . . ., x T) is a sequence of words describing the MR. The task is to match every word x i in text X with a possible slot. Note that for a data-text pair, not all slot-value pairs are mentioned in paired text, and not all words in text can be grounded to one of M slots in paired MR. However, some of unaligned words can be ground to one of K slots in the whole corpus. An example of alignments in a data-text pair is shown in Figure 1. The example displays differences between MR and text: contradiction (Rating:low corresponds to the text span:highly recommended), extra slots in text (EatType:restaurant). We add a special label NULL indicating words without any specific semantic annotation (e.g., stopwords). In the next section, we present our approach to address this task. Our proposed method is a local-to-global alignment (L2GA) model, as shown in Figure 2. It consists of two modules. The local model first encodes both description text X and its paired MR R using contextualized encoders, then acquires semantic alignments by computing similarities in between words and slot-value pairs presented in its paired MR R. As the input MR can be incomplete, a global model with a specific CRF layer is proposed to exploit dependencies among alignments over the entire corpus and therefore produces possible semantic labels for text spans not supported in the paired MR. Moreover, to incorporate the alignment guidance provided by the local model, a specific memory is integrated into CRF layer to make the final alignment decision. The local model tries to induce semantic labels for words in text X with respect to its paired input MR. Given a data-text pair (R, X), we can only assume that words in the description X are positively related for some slot-value pairs in R but the exact alignments are not provided. One possible way is to discover the fine-grained annotations (i.e., word alignments) from the coarse level supervisions (i.e., the similarity between a MR-text pair). Following , we formulate this task into a multi-instance learning problem . We first introduce the encoders for input MR R and description text X, then the alignment objectives to acquire the word level annotations for text X. MR Encoder: A slot-value pair r in MR can be treated as a short sequence w 1,..., w n by concatenating words in its slot and value. The word sequence is first represented as a sequence of word embedding vectors (v 1, . . ., v n) using a pre-trained word embedding matrix E w, and then passed through a bidirectional LSTM layer to yield the contextualized representations H = (h 1, . . ., h n). To produce a summary context vector, we adopt the same self-attention structure in to obtain the vector of slot-value pair c, due to the effectiveness of self-attention modules over variable-length sequences. where W s is a trainable parameter and β is the learned importance. We also embed each slot s i into a slot vector as where E z is a trainable slot embedding matrix. Sentence Encoder: For description X = (x 1, . . ., x T), each word x t is first embeded into vector e t by concatenating the word embedding and character-level representation generated with a group of convolutional neural network (CNNs). Then we feed the word vectors e 1,..., e T to a bidirectional LSTM to obtain contextualized vectors U = (u 1, . . ., u T). Alignment Objective: Our goal is to maximize the similarity score between the MR-text pair (R, X), and we will also learn the contribution of word-level annotations for words and slot-value pairs. Concretely, we first embed slot-value pairs in the input R = (r 1, ..., r M) into context vectors c 1,..., c M using the MR encoder defined in Eq.1. Similarly, we obtain the contextual vectors u 1,..., u T of description X using the sentence encoder defined in Eq.3. This similarity between MR-text pair is in turn defined on the top of the similarity scores among vector representations of slot-value pairs in R and words in description X as follows. where · refers inner product of two vectors. The function in Eq.4 aims to align each word with the best scoring slot-value pair. Note that each word x t is aligned with a slot-value pair r i if the similarity (i.e., inner product between two vectors) is larger than a threshold. To train the local alignment model, The loss function defined in Eq.5 is to encourage related MR R and description X to achieve higher similarity than other MR R = R and texts X = X: Since the data-text pairs are loosely corresponded, there exists text spans not supported by its noisy paired input. To induce semantic labels for those text spans, our proposed global alignment model is built on a CRF based sequence labeling framework which is capable of leveraging dependencies among alignments. Compared to conventional sequence labeling problem, our scenario differs in two aspects: i) lacking training labels for sequence labeling; ii) leveraging alignment information provided by the local alignment model. To overcome the issue of lacking word-level annotations, we first generate pseudo labels for words in texts by exact string matching, where conflicted matches are resolved by maximizing the total number of matched tokens . Based on the of dictionary matching, each word falls into one of three categories: 1) it belongs to an entity mention with one slot presented in its paired MR; 2) it belongs to an (unknown) entity where its slot is either not directly labeled using string matching or not represented in its paired MR; 3) it is marked as a non-entity 1. To allow inducing semantic labels for words with unknown types, we change the sequence paths in CRF layer. To incorporate semantic annotations learned by local model, particularly for text spans that are not directly recognized by string heuristics and mislabeled as an unknown entities (e.g., affordable in Figure 2), the alignments are treated as a soft memory to integrate into the CRF layer. Modified LSTM-CRF: In conventional LSTM-CRF based sequence labeling model , given the text description X = {x t} T t=1 and the pseudo labels Y = {y t} T t=1. We first obtain contextual representations U for words in description X using the Eq. 3, and context vector u t for word x t is decoded by a linear layer W c into the label space to compute the score P t,yt for label y t. On top of the model, a CRF layer is applied to capture the dependencies among predicted labels. We define the score of the predicted sequence, the score of the predicted (y 1, ..., y T) as: where, Φ yt,yt+1 is the transition probability from a label y t to its next label y t+1. Φ is a (K + 2) × (K +2) matrix, where K is the number of distinct labels (i.e., unique slots in the entire corpus). Two additional labels start and end are used (only used in the CRF layer) to represent the beginning and end of a sequence, respectively. The conventional CRF layer maximizes the probability of the only valid label sequence. However, there are entities with unknown types in our scenario (e.g., text spans restaurant and affordable Under review as a conference paper at ICLR 2020 in Figure 2 are unknown entities). We instead maximize the total probability of all possible label sequences by enumerating all the possible tags for entities with unknown types. The optimization goal is defined as: where Y X refers to all possible label sequences for X, and Y possible contains all the possible label sequences for entities with unknown type. Note that, if there are no entities with unknown type in description text X, it is equivalent to the conventional CRF. Integrate Local Alignment Clues: The local alignment model can provide alignment supervisions for words that are lexically different but semantically relevant to slot-value pairs in its paired MR. To incorporate the induced semantic labels provided by local alignment model, we design a specific memory into sequence labeling framework. Specially, for each word x t in description X, we select the most probable slot s i by computing similarity provided by local alignment model in Eq. 4, and compute the slot representation d t as follows where α t,i refers to the probability that word x t is related to slot s i in MR and z i is the slot embedding for slot s i defined in Eq. 2. We then utilize the alignment information d t to help the calculation of the prediction score P t in Equation 6. Concretely, we modified the Eq. 6 as following: where [,] refers to concatenation of two vectors. In this way, the alignments produced by local alignment model can act as a guidance to help inducing the labels of entities in texts. During training, we optimize the global model by minimizing negative log-likelihood p(Y |X) of the score defined in Eq. 8 for path Y given the text description X. We optimize the local and global model jointly using the following training loss: where L co is the alignment objective of local alignment model defined in Eq.5 and λ is a hyper parameter and we set λ to 1 according to the validation set. For inference, we apply Viterbi decoding to obtain the alignments for description texts by maximizing the score defined in Eq. 7. Our experiments are conducted on E2E challenge (b) dataset, which aims at verbalizing all information from the MR. It has 42,061, 4,672 and 4,693 MR-text pairs for training, validation and testing, respectively. Note that every input MR in this dataset has 8.65 different references on average. Our proposed model produces alignments based on the unique slots presented in the entire dataset. The unique slots in this dataset are {N ame, N ear, EatT ype, Rating, F ood, P rice, Area, F amilyF riendly}. It is difficult to evaluate the accuracy of alignment for the entire corpus, since the alignments are not provided in the original data. Due to the ambiguity of alignment boundaries (e.g., it is reasonable to tag all three words in price is low as Price or a single word low as Price), different alignment models have different alignment boundaries accordingly. Instead, alignments can be used to reproduce a refined MR by recovering slot-value pairs using the detected spans and its corresponding labels (e.g., word price is low and its label Price refers to a slot-value pair Price:low), more details in Appendix A.2. To make fair comparisons, we evaluate the alignments by its produced MR. The testset contains 630 unique input MRs, we randomly sample a reference for each MR, and recruited three human annotators to label the 630 data-text pairs. The annotators were required to refine original input MRs if reference text contains contradicted or unsupported facts 2. We calculate the precision and recall for the refined MR produced by alignment models with the annotated one. We compare our proposed alignment model with the following neural baselines: i) MIL , which refers to the local model. Note that each word is assigned to a slot if the semantic similarity defined in Eq.4 is larger than 0.1; ii) Distant LSTM-CRF , which is a dictionary based sequence labeling model for distant supervised name entity recognition (NER). We make adaptation by treating the paired MR as the dictionary to create initial training labels described in Section 3.2 and train a LSTM-CRF model based on the pseudo training data; iii) Modified LSTM-CRF, which is our proposed global model without leveraging local alignment information as described in Section 3.2. Table 1 presents the of our proposed method (L2GA) with other baselines. the MIL is the local alignment model, which can only leverage the information within a data-text pair. Therefore it is incapable of inducing potential alignments for text spans that are not supported by its paired MR. While our proposed method L2GA can exploit dependencies among alignments globally, therefore, improves the overall alignment performance (11.43% F1 improvement with respect to MIL). The other two methods are distant supervised sequence labeling approaches, which can be treated as simpler variations of our proposed global alignment model. The Distant LSTM-CRF performs worse than Modified LSTM-CRF which indicates the necessity of exploring all possible sequences in CRF layer for unknown entities. In this way, the model is able to induce a potential semantic labels for unknown type entities. Additionally, both Distant LSTM-CRF and Modified LSTM-CRF models utilize the information in its paired MR only in the creation of pseudo labels. Labels created by string matching is mislabeled as unmatched entities for text spans that are semantically equivalent but lexically different to some slot-value pairs (e.g., afforable is closely related to the slot-value pair Price:Cheap in Figure 2). While our proposed method can leverage the alignment information provided in its paired MR by the local alignment model simultaneously and therefore achieves substantial improvements. As our proposed method is target on learning the alignments in loosely related data-text pairs, we pick data-text pairs in testset where the human annotated MR contains additional or contradicted slot-value pairs compared to the original MR, and we report the performance of each method on the Noisy data-text pairs. Results in Table 1 shows that the performance of local model MIL decrease dramatically, while global models such as Modified LSTM-CRF and L2GA are less sensitive, which proves the necessity of using a global model in learning alignments for loosely related data-text pairs. Our proposed method L2GA outperforms the Modified LSTM-CRF in both settings. The further illustrate that both local and global models are essential for learning alignments in loosely related data-text pairs. We report detailed alignment F1 scores of our proposed method under each slot shown in Table 2. Our proposed L2GA achieves best in 4 out of 8 slots. The local model performs Under review as a conference paper at ICLR 2020 Table 3: Different combinations of local models with sequence labeling framework bad in EatType, which is one of the most common missing slot in the training set. The slot familyFriendly contains various expressions in corresponding texts, where Modified LSTM-CRF performs a lot worse than our proposed L2GA. The indicates the necessity of integrating alignment guidance from the local model. We also investigate different ways of incorporating local model with the sequence labeling framework. A straight forward way is to create new pseudo labels for sequence labeling framework using the alignments produced by local models and train a LSTM-CRF model based on the new training labels. Table 3 gives the . The of the separate model performs worse than our proposed L2GA, which indicates that accurate training labels are essential to sequence labeling. L2GA dynamically integrate the provided by the local model without introducing label noise for training, therefore achieves better . In this section, we provide an extrinsic evaluation by testing whether alignments can help neural generation. Neural generation models trained on noisy data-text pairs suffers from hallucination , where the generated texts produce contradicted or irrelevant facts with respect to its paired input. Alignments can produce a refined MR for each data-text pair, therefore, we can create a refined training corpus by applying our proposed method L2GA in training dataset. We use the new training corpus to train a sequence-to-sequence (S2S) generation model. To evaluate the correctness of generation, a well-crafted rule-based aligner built by is adopted to approximately reflect the semantic correctness. The error rate is calculated by matching the slot values in output texts containing missing or conflict slots in the realization given its input MR. The generation are shown in Table 4. Vanilla S2S model trained on loosely related data-text pairs performs poorly in generation correctness. After training on the corpus refined by our proposed L2GA method, S2S model can reduce the inconsistent errors in a large margin. The also indicates the value of studying alignments in the setting of loosely related data-text pairs, which can be of help to automatically reduce data noise in large datasets. 4.6 QUALITATIVE ANALYSIS Figure 3 gives the alignment produced by different models. We can see that local model MIL cannot induce the label for text spans kid friendly as it is contradicted with the slot-value pair FamilyFriendly:no. While global models can induce the semantic label for the text span kid friendly with the corresponding label FamilyFriendly. Moreover, the Modified LSTM-CRF has difficulty in labeling lexically different but semantically equivalent word highly. While L2GA Under review as a conference paper at ICLR 2020 L2GA: The Cricketers is a kid friendly restaurant that serves English food near All Bar One in the riverside area. It has a price range of 20-25 pounds and is a highly rated restaurant. Food Near Area Price Rating EatType The Cricketers is a kid friendly restaurant that serves English food near All Bar One in the riverside area. It has a price range of 20-25 pounds and is a highly rated restaurant. can dynamically integrate the alignment provided by the local model, therefore produce the semantic labels Rating for text span highly rated correctly. Previous work exploiting loosely aligned data and text corpora have mostly focused on discovering verbalisation spans for data units. These line of work usually follows a two stage paradigm: firstly, data units are aligned with sentences from related corpora using heuristics and then subsequently extra content is discarded in order to retain only text spans verbalising the data. use a measure of association between data units and words to obtain verbalisation spans. extract patterns from paths in dependency trees. One exception is , the induced alignments are used to guide the generation. Our work takes a step further to also induce alignments for text spans not supported by the noisy paired input with possible semantics. Our work is also related to previous work on extracting information from user queries with the backend data structure. Most of these approaches contain two steps. Initially, a separate model is applied to match the unstructured texts with relevant input records and then an extraction model is learned based on collected annotations. and train a language model on data records to identify related text spans in book description. Several approaches train a CRF based extractor to detect the related text spans . apply a generalized expectation criteria to learn alignments between database and the texts, and train the information extractor to induce semantic annotations for text spans. Compared to these work, our approach is an unified neural based alignment model which avoids the error propagation of each step. In this paper, we study the problem of learning alignments in loosely related data-text pairs. We propose a local-to-global framework which not only induces semantic correspondences for words that are related to its paired input but also infers potential labels for text spans that are not supported by its incomplete input. We find that our proposed method improves the alignment accuracy, and can be of help to reduce the noise in original training corpus. In the future, we will explore more challenging datasets with more complex data schema. Under review as a conference paper at ICLR 2020 are 300 and 100 respectively. The dimensions of trainable hidden units in LSTMs are all set to 400. We first pre-train our local model for 5 epochs and then train our proposed local-to-global model jointly with 10 epochs according to validation set. During training, we regularize all layers with a dropout rate of 0.1. We use stochastic gradient descent (SGD) for optimisation with learning rate 0.015. The gradient is truncated by 5. Given a MR-text pair (R, X) along with its induced alignments Y, our goal is to recover a refined MR R by making use alignments Y. Intuitively, values for several slots belong to string values (e.g., text span The Cricketers with semantic label Name), where the value is directly recovered by the corresponding text spans (e.g., Name:The Cricketers). In the E2E dataset, there are two slots with a string value (i.e., Name and Near). The rest of slots use categorical values. To recover for categorical values, we apply a simple retrieval based method. Specifically, we collect the text spans with the detect labels (i.e., slots) in the training corpus with its corresponding slot-value pair presented in the MR (e.g., text span kid friendly with slot FamilyFriendly). Since the MR can be inaccurate, text spans with a specific label might have multiple referring slot-value pairs (e.g., text span kid friendly has two options FamilyFriendly:yes and FamilyFriendly:no). We calculate the frequency of candidate slot-value pairs, and use the most frequent one (e.g., kid friendly is recovered to FamilyFriendly:yes as it co-occurs with FamilyFriendly:yes a lot more than FamilyFriendly:no).

We propose a local-to-global alignment framework to learn semantic correspondences from noisy data-text pairs with weak supervision

represent this text to find a much much longer article about it with say hundreds-thousands of words

We provide insightful understanding of sequence-labeling NER and propose to use two types of cross structures, both of which bring theoretical and empirical improvements.