Question: Represent the scientific article to find a short TLDR

Question: Effectively inferring discriminative and coherent latent topics of short texts is a critical task for many real world applications. Nevertheless, the task has been proven to be a great challenge for traditional topic models due to the data sparsity problem induced by the characteristics of short texts. Moreover, the complex inference algorithm also become a bottleneck for these traditional models to rapidly explore variations. In this paper, we propose a novel model called Neural Variational Sparse Topic Model (NVSTM) based on a sparsity-enhanced topic model named Sparse Topical Coding (STC). In the model, the auxiliary word embeddings are utilized to improve the generation of representations. The Variational Autoencoder (VAE) approach is applied to inference the model efficiently, which makes the model easy to explore extensions for its black-box inference process. Experimental onWeb Snippets, 20Newsgroups, BBC and Biomedical datasets show the effectiveness and efficiency of the model. With the great popularity of social networks and Q&A networks, short texts have been the prevalent information format on the Internet. Uncovering latent topics from huge volume of short texts is fundamental to many real world applications such as emergencies detection BID18, user interest modeling BID19, and automatic query-reply BID16. However, short texts are characteristic of short document length, a very large vocabulary, a broad range of topics, and snarled noise, leading to much sparse word co-occurrence information. Thus, the task has been proven to be a great challenge to traditional topic models. Moreover, the complex inference algorithm also become a bottleneck for these traditional models to rapidly explore variations. To address the aforementioned issue, there are many previous works introducing new techniques such as word embeddings and neural variational inference to topic models. Word embeddings are the low-dimensional real-valued vectors for words. It have proven to be effective at capturing syntactic and semantic information of words. Recently, many works have tried to incorporate word embeddings into topic models to enrich topic modeling BID5 BID7 BID22. Yet these models general rely on computationally expensive inference procedures like Markov Chain Monte Carlo, which makes them hard to rapidly explore extensions. Even minor changes to model assumptions requires a re-deduction of the inference algorithms, which is mathematic challenging and time consuming. With the advent of deep neural networks, the neural variational inference has emerged as a powerful approach to unsupervised learning of complicated distributions BID8 BID17 BID14. It approximates the posterior of a generative model with a variational distribution parameterized by a neural network, which allows back-propagation based function approximations in generative models. The variational autoencoder (VAE) BID8, one of the most popular deep generative models, has shown great promise in modeling complicated data. Motivated by the promising potential of VAE in building generative models with black-box inference process, there are many works devoting to inference topic models with VAE BID20 BID13 BID4. However, these methods yield the same poor performance in short texts as LDA.Based on the analysis above, we propose a Neural Variational Sparse Topic Model (NVSTM) based on a sparsity-enhanced topic model STC for short texts. The model is parameterized with neural networks and trained with VAE. It still follows the probabilistic characteristics of STC. Thus, the model inherit s the advantages of both sparse topic models and deep neural networks. Additionally, we exploit the auxiliary word embeddings to improve the generation of short text representations.1. We propose a novel Neural Variational Sparse Topic Model (NVSTM) to learn sparse representations of short texts. The VAE is utilized to inference the model effectively. 2. The general word semantic information is introduced to improve the sparse representations of short texts via word embeddings. 3. We conduct experiments on four datasets. Experimental demonstrate our model's superiority in topic coherence and text classification accuracy. The rest of this paper is organized as follows. First, we reviews related work. Then, we present the details of the proposed NVSTM, followed by the experimental . Finally, we draw our . Topic models. Traditional topic models and their extensions BID0 BID2 BID12 have been widely applied to many tasks such as information retrieval, document classification and so on. These models work well on long texts which have abundant word co-occurrence information for learning, but get stuck in short texts. There have been many efforts to address the data sparsity problem of short texts. To achieve sparse representations in the documenttopic and topic-term distributions, BID21 introduced a Spike and Slab prior to model the sparsity in finite and infinite latent topic structures of text. Similarly, BID10 proposed a dual-sparse topic model that addresses the sparsity in both the topic mixtures and the word usage. These models are inspired by the effect of the variation of the Dirichlet prior on the probabilistic topic models. There are also some non-probabilistic sparse topic models aiming at extracting focused topics and words by imposing various sparsity constraints. BID6 formalized topic modeling as a problem of minimizing loss function regularized by lasso. Subsequently, Zhu & Xing presented sparse topical coding (STC) by utilizing the Laplacian prior to directly control the sparsity of inferred representations. However, over complicated inference procedure of these sparse topic models has limited their applications and extensions. Topic Models with Word Embeddings. Since word embeddings can capture the semantic meanings of words via low-dimensional real-valued vectors, there have been a large number of works on topic models that incorporate word embeddings to improve topic modeling. BID5 proposed a new technique for topic modeling by treating the document as a collection of word embeddings and topics itself as multivariate Gaussian distributions in the embedding space. However, the assumption that topics are unimodal in the embedding space is not appropriate, since topically related words can occur distantly from each other in the embedding space. Therefore, BID7 proposed latent concept topic model (LCTM), which modeled a topic as a distribution of concepts, where each concept defined another distribution of word vectors. BID15 proposed Latent Feature Topic Modeling (LFTM), which extended LDA to incorporate word embeddings as latent features. Lately, BID22 proposed a novel correlated topic model using word embeddings, which is enable to exploit the additional word-level correlation information in word embeddings and directly model topic correlation in the continuous word embedding space. However, these models also have trouble to rapidly explore extensions. Neural Variational Inference for topic models. Neural variational inference is capable of approximating the posterior of a generative model with a variational distribution parameterized by a neural network BID8 BID17 BID14. The variational autoencoder (VAE), as one of the most popular neural variational inference approach, has shown great promise in building generative models with black-box inference process BID8. To break the bottleneck of over complicated inference procedure in topic models, there are many efforts devoting to inference topic models with VAE. BID20 presents auto-encoding variational Bayes (AEVB) based inference method for latent Dirichlet allocation (LDA), tackling the problems caused by the Dirichlet prior and component collapsing in AEVB. BID13 presents alternative neural approaches in topic modeling by providing parameterized distributions over topics. It allows training the topic model via back-propagation under the framework of neural variational inference. BID4 combines certain motivating ideas behind variations on topic models with modern techniques for variational inference to produce a flexible framework for topic modeling that allows for rapid exploration of different models. Nevertheless, aforementioned works are based on traditional LDA, thus bypass the sparsity problem of short texts. Drawing inspiration from the above analysis, we propose a novel neural variational sparse topic model NVSTM based on VAE for short texts, which combines the merits of neural networks and sparsity-enhanced topic models. In this section, we start from describing Sparse Topical Coding (STC). Based on it, we further propose Neural Variational Sparse Topic Model (NVSTM). Later, we focus on the discussion of the inference process for NVSTM. Firstly, we define that D = {1, ..., M} is a document set with size M, T = {1, ..., K} is a topic collection with K topics, V = {1, .., N} is a vocabulary with N words, and w d = {w d,1, .., w d,|I|} is a vector of terms representing a document d, where I is the index of words in document d, and w d,n (n ∈ I) is the frequency of word n in document d. Moreover, we denote β ∈ R N ×K as a topic dictionary for the whole document set with k bases, DISPLAYFORM0 K is the word code of word n in document d. To yield interpretable patterns, (θ, s, β) are constrained to be to be non-negative. In standard STC, each document and each word is represented as a low-dimensional code in topic space. Based on the topic dictionary β with K topic bases sampled from a uniform distribution, the generative process is described as follows: DISPLAYFORM0 STC reconstructs each observed word count from a linear combination of a set of topic bases, where the word code is utilized as the coefficient vector. To achieve sparse word codes, STC defines DISPLAYFORM0 The composite distribution is super-Gaussian: DISPLAYFORM1 With the Laplace term, the composite distribution tends to yield sparse word codes. For the same purpose, the prior distribution p(θ d) of sparse document codes is a Laplace prior Laplace(0, λ −1). Additionally, According to the above generative process, we have the joint distribution: DISPLAYFORM2 To simplify the calculation, the document code can be collapsed and later obtained via an aggregation of the individual word codes of all its terms. Although STC has closed form coordinate descent equations for parameters (θ, s, β), it is inflexible for its complex inference process. To address the aforementioned issue, we introduce black box inference methods into STC. We present NVSTM based on VAE and introduces word embeddings. As in BID1, we remove the document code and generate it via a simple aggregation of all sampled word codes among all topics: DISPLAYFORM0. Analogous to the generative process in STC, our model follows the generative story below for each document d: DISPLAYFORM1 The graphical representation of NVSTM is depicted in Figure 1. Different from STC, we replace the super-Gaussian with a uniform distribution. In the inference process, we adopt the variational posterior Laplace(s d,nk ; 0, σ d,nk (w d,n)) to approximate the intractable posterior p(s d,nk |w d,n) for the sparse word codes, where σ d,nk is the scale parameter of Laplace distribution. Therefore, in the above generative process, each word code vector is generated from the uniform prior distribution. The observed word count is sampled from Poisson distribution. Different from traditional STC, we replace the uniform distribution of the topic dictionary with a topic dictionary neural network. In the topic dictionary neural network, we introduce the word semantic information via word embeddings to enrich the feature space for short texts. The topic dictionary neural network is comprised of following:Word embedding layer (E ∈ R N ×300): Supposing the word number of the vocabulary is N, this layer devotes to transform each word to a distributed embedding representation. Here, we adopt the pre-trained embeddings by GloVe based on a large Wikipedia dataset 1. Given a word embedding matrix E, we map each word to a 300-dimensional embedding vector, which can capture subtle semantic relationships between words. Topic dictionary layers (β ∈ R N ×K): This layers aim at converting E to a topic dictionary similar to the one in STC. DISPLAYFORM2 where f is a multilayer perceptron. To conform to the framework of STC, we make a simplex projection among the output of topic dictionary neural network. We normalize each column of the dictionary via the simplex projection as follow: DISPLAYFORM3 The simplex projection is the same as the sparsemax activation function in BID11, which declares how the Jacobian of the projection can be efficiently computed, providing the theoretical base of its employment in a neural network trained with backpropagation. After the simplex projection, each column of the topic dictionary is promised to be sparse, non-negative and united. Based the above generative process, the traditional variational inference for the model is to minimize the follow optimization problem, which is a lower bound to the marginal log likelihood: DISPLAYFORM4 where q(s|γ) is approximate variational posterior, and γ is the variational parameter. In this paper, we employ the VAE to carry out neural variational inference for our model. Variational Autoencoder (VAE) is one of the most popular deep generative network. It is a black-box variational method which bridges the conceptual and language gap of neural networks and probability generative models. From neural network perspective, a variational autoencoder consists of an encoder network, a decoder network, and a loss function. In our model, the encoder network is to parametrize the approximate posterior q θ (s|w), which takes input as the observed word count to output the latent variable s with the variational parameters θ: DISPLAYFORM0 where f e (w d,n) is a multilayer perceptron acting on the word counts w d,n in document d, and logσ d,nk is the scale parameter of the approximate posterior, from which the word codes s d,nk are sampled. The decoder network outputs the observed data w with given s and the generative parameters φ, which is denoted as p φ (w|s, β). According to STC, we define the decoder network DISPLAYFORM1 DISPLAYFORM2 where f d is a multilayer perceptron. Based on VAE, we rewrite the ELBO as: DISPLAYFORM3 The first term is a regularizer that constraints the Kullback-Leibler divergence between the encoder's distribution distribution and the prior of the latent variables. The second term is the reconstruction loss, which encourages the decoder to reconstruct the data in minimum cost. We devote to differentiate and optimize the lower bound above with stochastic gradient decent (SGD). However, the gradient of the lower bound is tricky since the error is unable to back propagate through a random drawn variable s, which is a non-continuous and has no gradient. Similar to the standard VAE, we make a differentiable transformation, called reparameterization trick. We approximate s with an auxiliary noise variable ε ∼ U (−0.5, 0.5): DISPLAYFORM0 Through reparametrization, we can take s as a function with the parameter b deriving from the encoder network. It allows the reconstruction error to flow through the whole network. FIG0 presents the complete VAE inference process for NVSTM. Moreover, in order to achieve interpretable word codes as in STC, we constrain s to be non-negative, activation function on the output s of encoder. After apply the reparameterization trick to the variational lower bound, we can yield DISPLAYFORM1 where Θ represents the set of all the model. As explained above, the decoding term logp(w d,n |s d,nk, β nk) is the Poisson distribution, and β is generated by a topic dictionary neural network. After the differentiable transformation, the variation objective function can be computed in closed form and efficiently solved with SGD. The detailed algorithm is shown in Algorithm 1. To evaluate the performance of our model, we present a series of experiments below. The objectives of the experiments include: the qualitative evaluations: classification accuracy of documents and sparse ratio of latent representations; the qualitative inspection: the quality of extracted topics and document representations. Our evaluation is based on the four datasets:• 20Newsgroups: The classic 20 newsgroups dataset, which is comprised of 18775 newsgroup articles with 20 categories, and contains 60698 unique words 2.• Web Snippet: The web snippet dataset, which includes 12340 Web search snippets in 8 categories. We remove the words with fewer than 3 characters or whose document frequency less than 3 in the dataset. After the preprocessing, it contains 5581 unique words. 3.• BBC: It consists of 2225 BBC news articles from 2004-2005 with 5 classes. We only use the title and headline of each article. We remove the stop words and the words whose document frequency less than 3 in the dataset 4.• Biomedical: It consists of 20000 paper titles from 20 different MeSH in BioASQ's official website. We convert letters into lower case and remove the words whose document frequency less than 3 in the dataset. After preprocessing, there are 19989 documents with 20 classes 5.Statistics on the four datasets after preprocessing is reported in TAB0.We compare our model with five topic models: et al., 2003). A classical probabilistic topic model. We use the open source LDA implemented by collapsed Gibbs sampling 6. We use the default settings with iteration number n = 2000, the Dirichlet parameter for distribution over topics α = 0.1 and the Dirichlet parameter for distribution over words η = 0.01. DISPLAYFORM0 • STC (Zhu & Xing). A sparsity-enhanced topic model which has been proven to perform better than many existing models. We adopt the implementation of STC released by its authors 7. We set the regularization constants as λ = 0.2, ρ = 0.001 and the maximum number of iterations of hierarchical sparse coding, dictionary learning as 100.• NTM BID3. A recently proposed neural network based topic model, which has been reported to outperform the Replicated Softmax model 8. In NTM, the learning rate is 0.01 and the regularization factor is 0.001. During the pre-training procedure for all weight matrices, they are initialized with a uniform distribution in interval [-4*sqrt(6./(n visible+n hidden)),4*sqrt(6./(n visible+n hidden))], where n visible=784 and n hidden=500.• DocNADE BID9 ). An unsupervised neural network topic model of documents and have shown that it is a competitive model both as a generative model and as a document representation learning algorithm 9. In DocNADE, we choose the sigmoid activate function, the hidden size is 50, the learning rate is 0.01, the bath size is 64 and the max training number is 1000.• GaussianLDA BID5. A new technique for topic modeling by treating the document as a collection of word embeddings and topics itself as multivariate Gaussian distributions in the embedding space 10. We use default values for the parameters. Our model is implemented in Python via TensorFlow. For four datasets, we utilize the pre-trained 300-dimensional word embeddings from Wikipedia by GloVe, which is fixed during training. For each out-of-vocabulary word, we sample a random vector from a normal distribution in interval. We adopted ADAM optimizer for weight updating with an initial learning rate of 4e − 4 for four dataset. All weight matrices are initialized with a uniform distribution in interval [0, 1e − 5]. In practice, we found that our model is stable with the size of hidden layer, and set it to 500. To evaluate the effectiveness of the representation of documents learned by NVSTM, we perform text classification tasks on web snippet, 20NG, BBC and Biomedical using the document codes learned by topic models as the feature representation in a multi-class SVM. For each method, after obtaining the document representations of the training and test sets, we trained an classifier on the training set using the scikit-learn library. We then evaluated its predictive performance on the test set. On web snippet, we utilize 80% documents for training and 20% for testing. On the 20NG dataset, we keep 60% documents for training and 40% for testing, which is the same configuration as in BID10. For BBC and Biomedical dataset, we also keep 60% documents for training and 40% for testing. TAB2 report the classification accuracy under different methods with different settings on the number of topics among the four datasets. It clearly denotes that 1) In the four datasets, the NVSTM yields the highest accuracy. 2) In general, the neural network based NVSTM, NTM,DocNADE and GLDA generate better document representations than STC and LDA, demonstrating the representative advantage of neural networks in distributed word representations. 3) Sparse models NVSTM are superior to non-sparse models (DocNADE, NTM, GLDA and LDA) separately. It indicates that sparse topic models are more capable to extract topics from short documents. In this part, we quantitatively investigate the word codes and documents codes learned by our model. We compute the average word code as s n = 1 Dn d∈Dn s d,n over all documents that word n appears in. Table 4 shows the average word codes of some representative words learned by NVSTM and LDA in 8 categories of web snippet. For each category, we also present the topics learned by NVSTM in TAB3. We list top-9 words according to their probabilities under each topic. In Table 4, the illustrate that the codes discovered by NVSTM are apparently much sparser than those discovered by LDA. It tends to focus on narrow spectrum of topics and obtains discriminative and sparse representations of word. In contrast, LDA generates word codes with many non-zeros due to the data sparsity, leading to a confused topic distribution. Besides, in NVSTM, it is clear that each non-zero element in the word codes represents the topical meaning of words in corresponding position. The weights of these elements express their relationship with the topics. Noticed that there are words (e.g. candidates) have only a small range of topical meanings, indicating a narrow usage of those terms. While other words (e.g. hockey and marketing) tend to have a broad spectrum of topical meanings, denoting a general usage of those terms. Here, each document code is calculated as BID1. To demonstrate the quality of the learned representations by our model, we produce a t-SNE projection with for the document codes of the four datasets learned by our model in FIG3. For Web Snippet, we sample 10% of the whole document codes. For 20newsgroups and Biomedical, we sample 30% of the whole document codes. As for BBC, we present the whole document codes. It is obvious to see that all documents are clustered into distinct categories, which is equal to the ground truth number of categories in the four datasets. It proves the semantic effectiveness of the documents codes learned by our model. DISPLAYFORM0 We propose a neural sparsity-enhanced topic model NVSTM, which is the first effort in introducing effective VAE inference algorithm to STC as far as we know. We take advantage of VAE to simplify the inference process, which require no model-specific algorithm derivations. With the employing of word embeddings and neural network framework, NVSTM is able to generate clearer and semanticenriched representations for short texts. The evaluation demonstrate the effectiveness and efficiency of our model. Future work can include extending our model with other deep generative models, such as generative adversarial network (GAN).

a neural sparsity-enhanced topic model based on VAE

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We provide insightful understanding of sequence-labeling NER and propose to use two types of cross structures, both of which bring theoretical and empirical improvements.


Question: Represent the text to find a very short summary (1-2 sentences max).

Question: Neural Tangents is a library for working with infinite-width neural networks. It provides a high-level API for specifying complex and hierarchical neural network architectures. These networks can then be trained and evaluated either at finite-width as usual, or in their infinite-width limit. For the infinite-width networks, Neural Tangents performs exact inference either via Bayes' rule or gradient descent, and generates the corresponding Neural Network Gaussian Process and Neural Tangent kernels. Additionally, Neural Tangents provides tools to study gradient descent training dynamics of wide but finite networks. The entire library runs out-of-the-box on CPU, GPU, or TPU. All computations can be automatically distributed over multiple accelerators with near-linear scaling in the number of devices. In addition to the repository below, we provide an accompanying interactive Colab notebook at https://colab.sandbox.google.com/github/google/neural-tangents/blob/master/notebooks/neural_tangents_cookbook.ipynb

Represent the TLDR for finding the corresponding scientific article

Represent the short summary for finding the corresponding scientific article (long)

An unsupervised learning method that uses meta-learning to enable efficient learning of downstream image classification tasks, outperforming state-of-the-art methods.


Question: Represent the scientific article to find a short TLDR

Question: Symbolic logic allows practitioners to build systems that perform rule-based reasoning which is interpretable and which can easily be augmented with prior knowledge. However, such systems are traditionally difficult to apply to problems involving natural language due to the large linguistic variability of language. Currently, most work in natural language processing focuses on neural networks which learn distributed representations of words and their composition, thereby performing well in the presence of large linguistic variability. We propose to reap the benefits of both approaches by applying a combination of neural networks and logic programming to natural language question answering. We propose to employ an external, non-differentiable Prolog prover which utilizes a similarity function over pretrained sentence encoders. We fine-tune these representations via Evolution Strategies with the goal of multi-hop reasoning on natural language. This allows us to create a system that can apply rule-based reasoning to natural language and induce domain-specific natural language rules from training data. We evaluate the proposed system on two different question answering tasks, showing that it complements two very strong baselines – BIDAF (a) and FASTQA – and outperforms both when used in an ensemble. We consider the problem of multi-hop reasoning on natural language input. For instance, consider the statements Socrates was born in Athens and Athens belongs to Greece, together with the question Where was Socrates born? There are two obvious answers following from the given statements: Athens and Greece. While Athens follows directly from the single statement Socrates was born in Athens, deducing Greece requires a reader to combine both provided statements using the knowledge that a person that was born in a city, which is part of a country, was also born in the respective country. Most recent work that addresses such challenges leverages deep learning based methods BID41 BID29 BID38 BID30 BID18 BID21 BID17 BID8, capable of dealing with the linguistic variability and ambiguity of natural language text. However, the black-box nature of neural networks makes it hard to interpret the exact reasoning steps leading to a prediction (local interpretation), as well as the induced model (global interpretation).Logic programming languages like Prolog BID46, on the other hand, are built on the idea of using symbolic rules to reason about entities, which makes them highly interpretable both locally and globally. The capability to use user-defined logic rules allows users to incorporate external knowledge in a straightforward manner. Unfortunately, because of their reliance on symbolic logic, systems built on logic programming need extensive preprocessing to account for the linguistic variability that comes with natural language BID23.We introduce NLPROLOG, a system which combines a symbolic reasoner and a rule-learning method with pretrained sentence representations to perform rule-based multi-hop reasoning on natural language input.1 Like inductive logic programming methods, it facilitates both global as well as local interpretation, and allows for straightforward integration of prior knowledge. Similarly to deep learning based approaches, it can be applied to natural language text without the need to transforming it to formal logic. At the core of the proposed method is an external non-differentiable theorem prover which can take similarities between symbols into account. Specifically, we modify a Prolog interpreter to support weak-unification as proposed by BID39. To obtain similarities between symbols, we utilize sentence encoders initialized with pretrained sentence embeddings BID28 and then fine-tune these for a downstream question answering task via gradient-based optimization methods. Since the ing system contains non-differentiable components, we propose using Evolution Strategies (ES) BID9 as a gradient estimator BID47 for training the systemenabling us to fine-tune the sentence encoders and to learn domain-specific logic rules (e.g. that the relation is in is transitive) from natural language training data. This in a system where training can be trivially parallelized, and which allows to change the logic formalism by simply exchanging the external prover without the need for an intricate re-implementation as an end-to-end differentiable function. In summary, our main contributions are: a) we show how Prolog-like reasoning can be applied to natural language input by employing a combination of pretrained sentence embeddings, an external logic prover, and fine-tuning using Evolution Strategies, b) we extend a Prolog interpreter with weak unification based on distributed representations, c) we present Gradual Rule Learning (GRL), a training algorithm that allows the proposed system to learn First-Order Logic (FOL) rules from entailment, and d) we evaluate the proposed system on two different Question Answering (QA) datasets and demonstrate that its performance is on par with state-of-the-art neural QA models in many cases, while having different failure modes. This allows to build an ensemble of NLPROLOG and a neural QA model that outperforms all individual models. Our work touches in general on weak-unification based fuzzy logic BID39 and focuses on multi-hop reasoning for QA, the combination of logic and distributed representations, and theorem proving for question answering. Multi-hop Reasoning for QA. One prominent approach for enabling multi-hop reasoning in neural QA models is to iteratively update a query embedding by integrating information from embeddings of context sentences, usually using an attention mechanism and some form of recurrency BID41 BID29 BID38 BID30. These models have achieved state-of-the-art in a number of reasoning-focused QA tasks. BID18 employ a differentiable memory structure that is updated each time a new piece of information (usually a sentence) is processed. The memory slots can be used to track the state of various entities, which can be considered as a form of temporal reasoning. Similarly, the Neural Turing Machine BID17 and the Dynamic Memory Network BID21, which are built on differentiable memory structures, have been used to solve synthetic QA problems requiring multi-hop reasoning. BID8 modify an existing neural QA model to additionally incorporate coreference information provided by a coreference resolution model in a preprocessing step, which improves performance on QA problems requiring multi-hop reasoning. All of the methods above perform reasoning more or less implicitly by updating latent vector representations, which makes an unambiguous interpretation of the exact reasoning steps difficult. Additionally, it is not obvious how a strong prior, like user-defined inference rules, could be imposed on the respective reasoning procedures. Besides, many of them have been evaluated only on artificially generated data sets and thus it is unclear how they perform when on data that involves natural linguistic variability. Combination of FOL and Distributed Representations. Investigating the combination of formal logic and distributed representations has a long tradition, which is reviewed by BID4. Strongly related to our approach is the combination of Markov Logic Networks BID32, Probabilistic Soft Logic BID1, and word embeddings, which has been applied to Recognizing Textual Entailment (RTE) and Semantic Textual Similarity (STS) BID14 BID2, and improves upon baselines utilizing either only logic or only distributed representations. An area in which neural multi-hop reasoning models have been thoroughly investigated is Knowledge Base Completion BID6 BID5 BID26 BID7. While QA could be in principle modeled as a KB-completion task, the construction of a densely connected KB from text is far from trivial, due to the inherent ambiguity of natural language. Without any preprocessing, even the moderately sized QA tasks considered in this work would produce a very large and sparsely connected KB.Closest to our approach is the Natural Theorem Prover (NTP) BID33, which obtains the final proof score for a statement by constructing a neural network that represents all possible proofs. The model is trained end-to-end using backward chaining and a differentiable unification operator. Since the number of possible proofs grows exponentially with the number of facts and rules, NTPs cannot scale even to moderately sized knowledge bases, and are thus not applicable to natural language problems in its current form. We circumvent this issue by using a non-differentiable prover and fine-tune the model using Evolution Strategies. Theorem Proving for Question Answering. Our work is not the first to apply theorem proving to QA problems. BID0 employ a system based on Natural Logic to search a large KB for a single statement that entails the candidate answer. This is somewhat orthogonal to our approach, as we aim to learn rules that combine multiple statements to answer a question. More traditional systems like Watson BID12 or COGEX BID23 utilize an integrated theorem prover, but require a transformation of the natural language input to logical form. In the case of COGEX, this improves the accuracy of the underlying system by 30%, and increases its interpretability. While this work is similar in spirit, we greatly simplify the preprocessing step by replacing the transformation of natural language to logic with the simpler approach of taking Open Information Extraction (Open IE) BID10 textual patterns. BID11 propose the OPENQA system that utilizes a mixture of handwritten and automatically obtained operators that are able to parse, paraphrase and rewrite queries, which allows them to perform large-scale QA on KBs that include Open IE triples. While this work shares the same goal -answering questions using facts extracted by Open IE -we choose a completely different approach to address the problem of linguistic variablity and focus on the combination of multiple facts by learning logical rules. In the following, we review the relevant for the custom Prolog engine employed by our method. Specifically, we briefly introduce Prolog's backward chaining algorithm and unification procedure BID35. We assume basic knowledge of formal logic and logic programming. In a nutshell, a Prolog program consists of a set of rules in the form of Horn clauses DISPLAYFORM0 DISPLAYFORM1 ) the body of the rule. We call B the body size of the rule and rules with a body size of zero are named atoms (short for atomic formula).2 If an atom does not contain any variable symbols it is termed fact. As in related work BID39 BID19 ), we disregard negation and disjunction. The central procedure of Prolog is unification. It receives two atoms and attempts to find variable substitutions that make both atoms syntactically equal. For example, the input atoms country(Greece, Socrates) and country(X, Y) in the following variable substitution after unification: {X/Greece, Y /Socrates}.The proof algorithm of Prolog is called backward-chaining. It starts from a goal atom g and attempts to prove it by applying suitable rules, thereby generating subgoals that are proved next. To find applicable rules, it attempts to unify g with the heads of all available rules. If this unification succeeds, the ing variable substitutions are applied to the atoms in the rule body and each of those atoms becomes a new subgoal. For instance, the application of the rule country(X, Y) ⇐ born_in(Y, X) to the goal country(Greece, Socrates) would yield the subgoal born_in(Socrates, Greece). Then the process is repeated for all subgoals until no subgoal is left, i.e., until all subgoals have been unified with a fact. The of this procedure is a set of rule applications and variable substitutions called proof. Note, that the number of possible proofs is exponential in the number of predicate and entity symbols, as every rule might be used in the proof of each subgoal. Pseudo code for weak unification can be found in Appendix A.1 and we refer the reader to for more details. To apply standard Prolog to a NLP problem like QA, one would have to account for semantic similarities and ambiguities with extensive and error-prone preprocessing, e.g. when transforming the natural language input to logical form. Our aim is to apply Prolog to natural language question answering where the same entity or relation can have different natural language surface forms. Thus, we replace the equality-based unification operator with similarity-based weak unification BID39, which allows to unify two symbols x, y if they are sufficiently similar, as judged by a similarity function ∼ θ parameterized by θ. Then, the of unification also contains a proof success score S that is the of the symbols' similarity and the previous success score S: S = (x ∼ y, S), where ∈ {min, ·} is an aggregation function. The of backward-chaining with weak unification are (possibly) multiple proofs, each with an associated proof success score. NLPROLOG combines inference based on weak unification and distributed representations to allow reasoning on natural language statements. The natural language statements are first transformed to triples using Open IE BID10. The symbols occurring in these triples and in the rules are then embedded into a vector space, which in turn is used to estimate similarities between symbols. The ing similarity function is subsequently used to perform a proof and consequently obtain a proof success score S. The proof success score is then utilized as a training signal for ES. An illustration of the process can be found in FIG1, where we visualize the interplay of the different components for our running example. We embed symbols using an encoder E θ: F ∪ P → R d parametrized by θ for entity and predicate symbols, where d denotes the embedding size. The ing embeddings are used to induce ∼ θ: (F ∪ P) 2 → R: DISPLAYFORM0 where cos denotes the cosine similarity between two vectors. There are alternative similarity functions such as Euclidean distance or RBF kernel, but in preliminary experiments we found cosine simlarity to work more robustly. We use an encoder function that uses a embedding lookup table for predicate symbols and a different one for entities. All embeddings representing natural language phrases are populated with sentence vectors that were pretrained with SENT2VEC BID28. Additionally, we introduce a third lookup table for the predicate symbols of rules and goals, because semantics of goal or rule predicates might differ from the semantics of fact predicates even if they share the same surface form. For instance, the query (X, parent, Y) could be interpreted either as (X, is the parent of, Y) or as (X, has the parent, Y) which are fundamentally different things. We propose to fine-tune the similarity function on a downstream task by updating the symbol embeddings. As NLPROLOG involves the non-differentiable proof search step, we cannot apply backpropagation for optimization. Instead, we propose to employ Evolution Strategies in conjunction with Adam to estimate the weight updates. ES recently showed good for Reinforcement Learning problems in BID36; BID22. More formally, the parameter update is computed as: DISPLAYFORM0 where J(θ) is the reward obtained by θ, σ t J is the standard deviation of all rewards obtained at time t as proposed by BID22, and α t are adaptive learning rates selected by ADAM BID20. The standard deviation σ of the distribution that is generating the perturbations is treated as a hyperparameter. We train NLPROLOG with ES using a learning from entailment setting BID25, in which the model is trained to decide whether a Prolog program R entails the truth of a candidate triple c. The objective of the model is to assign high probabilities p(c|R; θ) to true candidate triples and low probabilities to false triples. To achieve this, we model the reward as J(θ) = yp(c|R; θ), where y ∈ {−1, 1} is the gold label. To estimate p(c|R; θ), we exhaustively search for all proofs for the triple c, up to a given depth D which we treat as a hyperparameter. This search yields a number of proofs each with a success score S i. We set p(c|R; θ) to be the maximum of these scores S max = max i S i. The reasoning process of NLPROLOG crucially depends on rules that describe the relations between predicates. While it is possible to write down rules in natural language, this approach is hardly scalable. Thus, we follow BID33 and use rule templates to perform Inductive Logic Programming (ILP) BID24 which allows NLPROLOG to learn rules from training data. For this, a user has to define a set of rules with a given structure as input. Then, NLPROLOG randomly initializes the predicates of these rules. For instance, to induce a rule that can model transitivity, one would add a rule template of the form p 1 (X, Z) ⇐ p 2 (X, Y) ∧ p 3 (Y, Z), and NLPROLOG would instantiate multiple rules with randomly initialized embeddings for p 1, p 2, and p 3. The exact number and structure of the rule templates is treated as a hyper-parameter. If not explicitly stated otherwise, the experiments were performed with the same set of rule templates containing two rules for each of the forms DISPLAYFORM0 In preliminary experiments, we observed that unification with such randomly initialized embeddings always leads to a stark drop of the proof success score. This implies in turn that proofs involving rule templates rarely yield the highest score S max and thus, might have no impact on the value of the reward function. Due to that, the expected gradient estimate for the rule embeddings is zero, and they will remain close to their initialization. The main reason for this behavior is that the monotonicity of the aggregation functions implies that each sub-goal created by an application of a rule will only decrease the success score of the proof. Thus, all other things being equal, rules with a small number of body atoms (and facts in particular) will be preferred over more complex rules with a higher number of body atoms. Note that this problem is particularly severe in our setting where rules are initialized randomly while the remaining predicate symbols are instantiated using pretrained sentence embeddings. We propose a Gradual Rule Learning (GRL) algorithm which counteracts this effect during training. GRL segments the training process into B max + 1 phases, where B max is the maximum body size of all available rules. In the k-th phase of GRL, only proofs which employ at least one rule with a body size of B max + 1 − k are considered to estimate p(t|F ; θ). Thus, it is guaranteed that in each training step in phase k at least one rule with a body size of B max + 1 − k receives a training signal. We evaluate our method on two different QA data sets: BABI-1K-STATIC and different subsets of WIKIHOP BID44. The used hyperparameter configurations can be found in Section B. We evaluate on different subsets of WIKIHOP BID44, each containing a single query predicate. We consider the predicates publisher, developer, and country, because their semantics ensure that the annotated answer is unique and they contain a relatively high amount of questions that are annotated as requiring true multi-hop reasoning. For publisher, this yields 509 training and 54 validation questions, for developer 267 and 29, and for country 742 and 194. As the test set of WIKIHOP is not publicly available, and splitting the small train set would lead to a far too small validation set, we report scores for the validation set and refrain from hyperparameter optimization and early stopping. Each data point consists of a query p(q, X) where q is some query entity, X is the entity that has to be predicted, C is a list of candidates entities, a ∈ C is an answer entity and p ∈ {publisher, developer, country} is the query predicate. In addition, every query is accompanied by a set of support documents which can be used to decide which candidate is the correct answer. To transform the support documents to natural language triples, we use the Open IE system MINIE BID16. We use the publicly available version of MINIE 3 in the dictionary mode, and use a list of all WIKIHOP candidate entities as our dictionary of multi-token entities. Following BID44, we use the two neural QA models BIDAF BID37 and FASTQA BID42 as baselines for the selected predicates of WIKIHOP. We employ the implementation provided by the QA framework JACK 4 BID43 with the same hyperparameters as used by BID44 and train a separate model for each predicate. In order to compensate for the fact that both models are extractive QA models which cannot make use of the candidate entities, we additionally evaluate modified versions which transform both the predicted answer and all candidates to vectors using the wiki-unigrams model 5 of SENT2VEC BID28. Consequently, we return the candidate entity which has the highest cosine similarity to the predicted entity. We utilize a subset of to study the behavior of NLPROLOG in a very controlled environment. Note however, that the experiments on WIKIHOP are much more significant, as they involve natural linguistic variability. BABI-1K-STATIC consists of the tasks QA4,QA15,QA17, and QA18 from the BABI suite, each containing 1,000 training and 1,000 testing questions. These tasks were selected because, unlike the other tasks of BABI, they do not require reasoning about a dynamically changing world state, which is not supported by the current implementation of NLPROLOG. We automatically transform all statements and queries of the respective tasks to triples and use the ing KB as input to NLPROLOG. We train and evaluate on each problem individually. The tasks QA4 and QA15 require entities as an output, thus we consider every entity that occurs at least once in any problem of the task as a candidate for all problems. Tasks QA17 and QA18 are binary classification tasks, and thus we determine the optimal threshold on the training set, after the training of NLPROLOG has finished. We refrain from systematically comparing on the individual BABI tasks to competing methods like BID38; BID29; BID8; BID18; BID41, since our non-negligible preprocessing and evaluation on only four out of 20 tasks does not allow us to match the relevant evaluation protocols. We therefore utilize BABI-1K-STATIC only for ablation experiments, but note that NLPROLOG achieves similar or better accuracy values than the mentioned methods in all instances we studied, except on QA4. All questions of WIKIHOP and some of BABI-1K-STATIC include a set of answer candidates C. For those cases, we modify our reward function to leverage this information, taking inspiration from Bayesian Personalized Ranking BID31: DISPLAYFORM0 where a ∈ C is the true answer. We observed that this reward function does not work well with the minimum aggregation function. Therefore, we employ this modified reward only when using the product aggregation and utilize the reward described in Section 4.2 with the minimum aggregation. The for the selected predicates of WIKIHOP can be found in Table 1. While there is no single best performing model, NLPROLOG is outperformed by at least one neural QA model on every predicate. For country, this only holds true when considering the versions of the neural models that have been augmented to consider candidate entities. For all three predicates, only a single transitive rule is utilized across all validation questions. Since we observe a more diverse set of induced rules on BABI-1K, we partly attributed this lack of diverse rules to the multi-hop characteristic of the WIKIHOP data. It seems that NLPROLOG struggles to find meaningful rules for the predicates developer and publisher, which leads to very few proofs involving rules on the development set: 1 out of 54 for publisher and 2 out of 29 for developer, compared with 159 out of 194 for country. We partially attribute this to the fact, that the semantics of country suggest a straightforward rule (transitivity of location), which is not true for developer or publisher. Additionally, the annotations regarding the neccessity of multi-hop reasoning provided for the validation set BID44 suggest that publisher and developer contain significantly fewer training examples involving multi-hop reasoning. Exemplary proofs generated by NLPROLOG on the predicates developer and country can be found in Fig. 3. As we are especially interested in assessing the capability for multi-hop reasoning, we additionally evaluate on a subset of problems which have been unanimously labelled as requiring multi-hop reasoning. On this subset of the development data, which is denoted as Countryhop (mh) in Table 1, NLPROLOG outperforms all other single models. If the proof of NLPROLOG producing the prediction does not employ any rules, the prediction is essentially the of performing a simple nearest neighbor search among the embeddings of all fact triples. We hypothesize that the neural QA models FASTQA and BIDAF are better suited for finding the most predictive single support statement, which motivates an ensemble of a neural QA model and NLPROLOG. We built a system which predicts the output of NLPROLOG when it used at Table 1: Accuracy scores in percent for different predicates on the development set of WIKIHOP. country (mh) only considers the problems that have been unanimously judged to require multi-hop reasoning. The upper part lists single models and the bottom part ensembles.least one rule and the output of the neural QA model otherwise. This allows to employ the multihop reasoning of NLPROLOG when possible and to utilize the pattern matching capabilities of the neural QA models in the remaining cases. The for these ensembles are given in the bottom part of Table 1. In all instances, ensembling a neural QA model with NLPROLOG improved upon all single models, indicating that they complement each other well. We analyze reasons for the success of this ensembling strategy in Section 5.6. We conducted an extensive error analysis, in which we manually classified all errors of NLPROLOG on the selected WIKIHOP predicates into predefined categories. The are depicted in FIG2.For the developer predicate, in the majority of cases, errors were caused by OPENIE during the fact extraction step: in all but one case, OPENIE did not produce the necessary facts, or a necessary facts was not stated in the support text, or there were multiple correct candidates and NLPROLOG selected the wrong one. As a consequence, for both the publisher and developer predicates, the majority of queries would not answerable, even when the necessary rules were correctly induced. The predictive accuracy was significantly higher for the country predicate, where errors were mostly due to entitities not having SENT2VEC embeddings and a few missing rules. FIG2 indicates that FASTQA can produce the correct answer, even if crucial information is missing from the supporting documents. To analyze this further, we evaluated FASTQA and NLPROLOG on a test set of the country predicate in which all documents mentioning the query entity were removed. Remarkably, the accuracy of FASTQA increased by approximately 1%, while the accuracy of NLPROLOG decreased by approx. 11%.Furthermore, we evaluated both FASTQA and NLPROLOG on the hard subset of country as defined by BID40: on these 62 problems which cannot be solved with a simple heuristic, NLPROLOG achieved an accuracy of 51.61%, as opposed to 46.77% by FASTQA.We conjecture that -besides NLPROLOG's multi-hop reasoning capability -this is one reason why the neural models and NLPROLOG complement each other nicely: neural models can compensate for missing information, while NLPROLOG is less susceptible for spurious correlations between the query and supporting texts. The complementary nature of both approaches is further supported by the error analysis, described in Fig. 4. We perform experiments on BABI-1K-STATIC to investigate the effect of the GRL training procedure. Specifically, we perform experiments on BABI-1K-STATIC with only the last phase of GRL (i.e. training without GRL), with the last and the penultimate phase, and with all three phases, corresponding to full GRL as we limit the rule complexity to two body atoms in rule templates. To maintain comparability between runs, we keep the number of SGD steps constant across all experiments. refers to a crucial information missing due to an error of the OpenIE tool, A(mbiguous) means that multiple candidates were correct and NLProlog chose one which was not labeled as the correct one, while I(nfo) refers to problems that are not answerable with the provided support texts. R(ule) means that a required rule was not induced, and E(mbedding) implies that the answer was correctly deduced but received a lower score than an erroneous deduction of another candidate. Additionally, we experiment with manually defined rules, which we deem sufficient for solving each of the four tasks. For these, we report before and after training, as well as for a run without any rule templates. The accuracy scores for all experiments on BABI are provided in TAB2.To assess the impact of the choice of rule templates, we evaluate NLPROLOG on bAbI-1k-static with a different set of rule templates containing two rules of the form p 1 (X, Y) ⇐ p 2 (X, Y), four with the form p 1 (X, Y) ⇐ p 2 (Y, X) and another four for DISPLAYFORM0 Clearly, the full GRL is necessary to obtain acceptable in any of the problems. Interestingly, phase 1 of GRL does not contribute anything for QA4, which is perfectly solvable using only rules of body size 1. On the other hand, QA15 and QA18 both require a rule of body size 2, which makes phase 1 strongly improve the . Only for QA17 the are inconclusive. Nevertheless, this indicates that GRL works as intended, with the earlier phases encouraging the induction of rules with a higher number of conjuncts in the body. The using manually defined rules suggest that even when sufficient rules are provided, training with ES is helpful nevertheless. Interestingly, the model using no rules is able to solve over 90% of the problems in QA18, indicating that this problem is not well suited for evaluating reasoning capabilities of QA models. Using ten instead of six templates leads to worse performance on all BABI problems but QA15, which is solved perfectly with a much faster convergence rate. This indicates that the choice of rule templates might be an important hyperparameter which should be optimized for a given problem. We have developed NLPROLOG, a system that is able to perform rule-based reasoning on natural language input, and can learn domain-specific natural language rules from training data. To this end, Figure 3: Example proof trees generated by NLPROLOG. Each of the two trees shows an application of a transitive rule, the first for the predicate developer and the second for the predicate country. The rule templates are displayed with the most similar predicate. Note the noise introduced by the Open IE process, e.g. QUANT_0_1 and that entities and predicates do not need to match exactly.we have proposed to combine a symbolic prover with pretrained sentence embeddings and to train the ing system with Evolution Strategies. We have evaluated NLPROLOG on two different QA tasks, showing that it can learn domain-specific rules and produce predictions which complement those of the two strong baselines BIDAF and FASTQA. This allows to build an ensemble of a baseline and NLPROLOG which outperforms all single models. While we have focused on a subset of First Order Logic in this work, the expressiveness of NL-PROLOG could be extended by incorporating a different symbolic prover. For instance, a prover for temporal logic BID27 ) would allow to model temporal dynamics in natural language and enable us to evaluate NLPROLOG on the full set of BABI tasks. We are also interested in incorporating future improvements of symbolic provers, Open IE systems and pretrained sentence representations to further enhance the performance of NLPROLOG. To study the performance of the proposed method without the noise introduced by the Open IE step, it would be useful to evaluate it on tasks like knowledge graph reasoning. Additionally, it would be interesting to study the behavior of NLPROLOG in the presence of multiple WIKIHOP query predicates. else if x is f (x 1, . . ., x n), y is f (y 1, . . ., y n), and f ∼ f ≥ λ then S:= S ∧ f ∼ f return unify(x 1 :: . . . :: x n, y 1 :: . . . :: y n, θ, S) end else if x is p(x 1, . . ., x n), y is p (y 1, . . ., y n), and p ∼ p ≥ λ then S:= S ∧ f ∼ f return unify(x 1 :: . . . :: x n, y 1 :: . . . :: y n, θ, S) end else if x is x 1::...:: x n and y is y 1::...:: y n then (θ, S):= unify(x 1, y 1, θ, S) return unify(x 2 :: . . . :: x n, y 2 :: . . . :: y n, θ, S) end else if x is empty list and y is empty list then return (θ, S) else return (failure, 0) fun unify_var (v, o, θ, S) if {v/val} ∈ θ then return unify(val, o, θ, S) else if {o/val} ∈ θ then return unify(var, val, θ, S) else return ({v/o} + θ, S) Algorithm 1: The weak unification algorithm in Spyrolog without occurs check A.2 RUNTIME OF PROOF SEARCHThe worst case complexity vanilla logic programming is exponential in the depth of the proof BID34. However, in our case this is a particular problem because weak unification requires the prover to attempt unification between all entity/predicate symbols. To keep things tractable, NLPROLOG only attempts to unify symbols with a similarity greater than some user-defined threshold λ. Furthermore, in the search step for one statement q, for the rest of the search, λ is set to λ:= max(λ, S) whenever a proof for q with success score S is found. Due to the monotonicity of the employed aggregation functions, this allows to prune the search tree without losing the guarantee to find the proof yielding the maximum success score. We found this optimization to be crucial to make the proof search scale for the studied wikihop predicates. On BABI-1K we optimize the embeddings of predicate symbols of rules and query triples, as well as of entities. WIKIHOP has a large number of unique entity symbols and thus, optimizing their embeddings is prohibitive. Thus, we only train the predicate symbols of rules and query triples on this data set. The embeddings for entities and predicate symbols of fact and query triples are initialized using the WIKI-UNIGRAMS model of SENT2VEC, while the embeddings of rule predicates are intialized by uniformly sampling from the interval [− 1 √ 600 DISPLAYFORM0]. All experiments were performed with the same set of rule templates containing two rules for each of the forms p(X, Y) ⇐ q(X, Y), p(X, Y) ⇐ q(Y, X) and p(X, Z) ⇐ q(X, Y) ∧ r(Y, Z) and set the similarity threshold λ to 0.3. At each optimization step, we evaluate 100 perturbations sampled from N (0, 0.04) on a mini-batch of 16 training problems and use all of the directions in the generation of the next weight vector. If not stated otherwise, we use GRL with three phases training for 500 mini-batches in each phase. For the predicates publisher and developer, we used 1,000 mini-batches in the final phase of GRL. To further encourage rule usage, we use the minimum aggregation function in all but the last phases of GRL, in which we switch to product. Figure 4: Venn diagram of the error sets per predicate.

We introduce NLProlog, a system that performs rule-based reasoning on natural language by leveraging pretrained sentence embeddings and fine-tuning with Evolution Strategies, and apply it to two multi-hop Question Answering tasks.

Represent the TLDR for finding the corresponding scientific article

We propose neural cascades, a simple and trivially parallelizable approach to reading comprehension, consisting only of feed-forward nets and attention that achieves state-of-the-art performance on the TriviaQA dataset.


Question: Represent the article for finding a single-sentence TLDR

Question: Training labels are expensive to obtain and may be of varying quality, as some may be from trusted expert labelers while others might be from heuristics or other sources of weak supervision such as crowd-sourcing. This creates a fundamental quality-versus-quantity trade-off in the learning process. Do we learn from the small amount of high-quality data or the potentially large amount of weakly-labeled data? We argue that if the learner could somehow know and take the label-quality into account, we could get the best of both worlds. To this end, we introduce “fidelity-weighted learning” (FWL), a semi-supervised student-teacher approach for training deep neural networks using weakly-labeled data. FWL modulates the parameter updates to a student network, trained on the task we care about on a per-sample basis according to the posterior confidence of its label-quality estimated by a teacher, who has access to limited samples with high-quality labels. "All samples are equal, but some samples are more equal than others." -Inspired by George Orwell quote, The success of deep neural networks to date depends strongly on the availability of labeled data and usually it is much easier and cheaper to obtain small quantities of high-quality labeled data and large quantities of unlabeled data. For a large class of tasks, it is also easy to define one or more so-called "weak annotators" BID10, additional (albeit noisy) sources of weak supervision based on heuristics or weaker, biased classifiers trained on e.g. non-expert crowd-sourced data or data from different domains that are related. While easy and cheap to generate, it is not immediately clear if and how these additional weakly-labeled data can be used to train a stronger classifier for the task we care about. Assuming we can obtain a large set of weakly-labeled data in addition to a much smaller training set of "strong" labels, the simplest approach is to expand the training set simply by including the weakly-supervised samples (all samples are equal). Alternatively, one may pretrain on the weak data and then fine-tune on strong data, which is one of the common practices in semi-supervised learning. We argue that treating weakly-labeled samples uniformly (i.e. each weak sample contributes equally to the final classifier) ignores potentially valuable information of the label quality. Instead, we introduce Fidelity-Weighted Learning (FWL), a Bayesian semi-supervised approach that leverages a small amount of data with true labels to generate a larger training set with confidence-weighted weakly-labeled samples, which can then be used to modulate the fine-tuning process based on the fidelity (or quality) of each weak sample. By directly modeling the inaccuracies introduced by the weak annotator in this way, we can control the extent to which we make use of this additional source of weak supervision: more for confidently-labeled weak samples close to the true observed data, and less for uncertain samples further away from the observed data. In this section, we describe FWL. We assume we are given a large set of unlabeled data samples, a heuristic labeling function called the weak annotator, and a small set of high-quality samples labeled by experts, called the strong dataset, consisting of tuples of training samples x i and their true labels y i, i.e. D s = {(x i,y i)}. We consider the latter to be observations from the true target function that we are trying to learn. We use the weak annotator to generate labels for the unlabeled samples. Generated labels are noisy due to the limited accuracy of the weak annotator. This gives us the weak dataset consisting of tuples of training samples x i and their weak labelsỹ i, i.e. D w = {(x i,ỹ i)}. Note that we can generate a large amount of weak training data D w at almost no cost using the weak annotator. In contrast, we have only a limited amount of observations from the true function, i.e. |D s | |D w |. Step 1: Pre-train student on weak data, Step 2: Fit teacher to observations from the true function, and Step 3: Fine-tune student on labels generated by teacher, taking the confidence into account. Red dotted borders and blue solid borders depict components with trainable and non-trainable parameters, respectively. Our proposed setup comprises a neural network called the student and a Bayesian function approximator called the teacher. The training process consists of three phases which we summarize in FIG0.Step 1 Pre-train the student on D w using weak labels generated by the weak annotator. The main goal of this step is to learn a task dependent representation of the data as well as pretraining the student. The student function is a neural network consisting of two parts. The first part ψ learns the data representation and the second part φ performs the prediction task (e.g. classification). Therefore the overall function isŷ = φ(ψ(x i)). The student is trained on all samples of the weak dataset D w = {(x i,ỹ i)}. For brevity, in the following, we will refer to both data sample x i and its representation ψ(x i) by x i when it is obvious from the context. Step 2 Train the teacher on the strong data (ψ(x j), y j ) ∈ D s represented in terms of the student representation ψ and then use the teacher to generate a soft dataset D sw consisting of sample,predicted label, confidence for all data samples. We use a Gaussian process as the teacher to capture the label uncertainty in terms of the student representation, estimated w.r.t the strong data. A prior mean and covariance function is chosen for GP. The learned embedding function ψ(·) in Step 1 is then used to map the data samples to dense vectors as input to the GP. We use the learned representation by the student in the previous step to compensate lack of data in D s and the teacher can enjoy the learned knowledge from the large quantity of the weakly annotated data. This way, we also let the teacher see the data through the lens of the student. The GP is trained on the samples from D s to learn the posterior mean m post (used to generate soft labels) and posterior co-variance K post (., .) (which represents label uncertainty) BID17. We then create the soft dataset D sw = {(x t,ȳ t)} using the posterior GP, input samples x t from D w ∪ D s, and predicted labelsȳ t with their associated uncertainties as computed T (x t) = g(m post (x t)) and Σ(x t) = h(K post (x t,x t)). The generated labels are called soft labels. Therefore, we refer to D sw as a soft dataset. g transforms the output of GP to the suitable output space. For example in classification tasks, g would be the softmax function to produce probabilities that sum up to one. For multidimensional-output tasks where a vector of variances is provided by the GP, the vector K post (x t,x t) is passed through an aggregating function h to generate a scalar value for the uncertainty of each sample. Note that we train GP only on the strong dataset D s but then use it to generate soft labelsȳ t = T (x t) and uncertainty Σ(x t) for samples belonging to D sw = D w ∪D s.Step 3 Fine-tune the weights of the student network on the soft dataset, while modulating the magnitude of each parameter update by the corresponding teacher-confidence in its label. The student network of Step 1 is fine-tuned using samples from the soft dataset D sw = {(x t,ȳ t)} wherē y t = T (x t). The corresponding uncertainty Σ(x t) of each sample is mapped to a confidence value, and this is then used to determine the step size for each iteration of the stochastic gradient descent (SGD). So, intuitively, for data points where we have true labels, the uncertainty of the teacher is almost zero, which means we have high confidence and a large step-size for updating the parameters. However, for data points where the teacher is not confident, we down-weight the training steps of the student. This means that at these points, we keep the student function as it was trained on the weak data in Step 1. The weak annotator, i.e. the unsupervised method used for annotating the unlabeled data. Full Supervision Only, i.e. the student trained only on strong labeled data (Ds). Weak Supervision Only, i.e. the or the student trained only on weakly labeled data (Dw).NN W/S + Weak Supervision + Oversampled Strong Supervision, i.e. thestudent trained on samples that are alternately drawn from Dw without replacement, and Ds with replacement. Since |Ds| |Dw|, it oversamples the strong data. Weak Supervision + Fine Tuning, i.e. the student trained on weak dataset Dw and fine-tuned on strong dataset Ds. NN W ω →NN S The student trained on the weak data, but the step-size of each weak sample is weighted by a fixed value 0 ≤ ω ≤ 1, and fine-tuned on strong data. As an approximation for the optimal value for ω, we have used the mean of η2 of our model (below). The student trained on the weakly labeled data and fine-tuned on examples labeled by the teacher without taking the confidence into account. This baseline is similar to BID16.More specifically, we update the parameters of the student by training on D sw using SGD: DISPLAYFORM0 where l(·) is the per-example loss, η t is the total learning rate, N is the size of the soft dataset D sw, w w w is the parameters of the student network, and R is the regularization term. We define the total learning rate as η t = η 1 (t)η 2 (x t), where η 1 (t) is the usual learning rate of our chosen optimization algorithm that anneals over training iterations, and η 2 (x t) is a function of the label uncertainty Σ(x t) that is computed by the teacher for each data point. Multiplying these two terms gives us the total learning rate. In other words, η 2 represents the fidelity (quality) of the current sample, and is used to multiplicatively modulate η 1. Note that the first term does not necessarily depend on each data point, whereas the second term does. We propose η 2 (x t) = exp[−βΣ(x t)] to exponentially decrease the learning rate for data point x t if its corresponding soft labelȳ t is unreliable (far from a true sample). In practice, when using mini-batches, we implement this by multiplying the loss of each example in the batch by its fidelity score and average over these fidelity-weighted losses in the batch when calculating the batch gradient based on that loss. β is a positive scalar hyper-parameter that controls the contribution of weak and strong data to the training procedure. A small β in a student which listens more carefully to the teacher and copies its knowledge, while a large β makes the student pay less attention to the teacher, staying with its initial weak knowledge. Hence, β gives a handle to control the bias-variance trade-off. In Appendix A, we apply FWL to a one-dimensional toy problem to illustrate its various steps. In this section, we apply FWL to document ranking task and evaluate its performance compared to the baselines presented in TAB1. Document Ranking is the core information retrieval problem and is challenging as the ranking model needs to learn a representation for long documents and capture the complex notion of relevance between queries and documents. Furthermore, the size of publicly available datasets with query-document relevance judgments is unfortunately quite small (∼ 250 queries). We employ a state-of-the-art pairwise neural ranker architecture as the student BID3 in which the ranking is cast as a regression task. Given each training sample x as a triple of query q, and two documents d + and d −, the goal is to learn a function F: {< q,d +,d − >} → R, which maps each data sample x to a scalar output value y indicating the probability of d + being ranked higher than d − with respect to q. The student follows the architecture proposed in BID3. The first layer of the network, i.e. representation learning layer ψ: {< q,d DISPLAYFORM0 − >} → R m maps each input sample to an mdimensional real-valued vector. In general, besides learning embeddings for words, function ψ learns to compose word embedding based on their global importance in order to generate query/document embeddings. The representation layer is followed by a simple fully-connected feed-forward network with a sigmoidal output unit to predict the probability of ranking d + higher than d −. The general schema of the student is illustrated in FIG1. More details are provided in Appendix C.1.The teacher is implemented by clustered GP algorithm. See Appendix C.2 for more details. The weak annotator is BM25 BID11 ), a well-known unsupervised method for scoring query-document pairs based on statistics of the matched terms. More details are provided in Appendix C.3. Description of the data with weak labels and data with true labels as well as the setup of the document-ranking experiments is presented in Appendix C.4 in more details. Results and Discussions. We conducted k-fold cross-validation on D s (the strong data) and report two standard evaluation metrics for ranking: mean average precision (MAP) of the top-ranked 1,000 documents and normalized discounted cumulative gain calculated for the top 20 retrieved documents (nDCG@20). TAB2 shows the performance on both datasets. As can be seen, FWL provides a significant boost on the performance over all datasets. In the ranking task, the student is designed in particular to be trained on weak annotations BID3, hence training the network only on weak supervision, i.e. NN W performs better than NN S. This can be due to the fact that ranking is a complex task requiring many training samples, while relatively few data with true labels are available. Alternating between strong and weak data during training, i.e. NN S + /W seems to bring little (but statistically significant) improvement. However, we can gain better by the typical fine-tuning strategy, NN W→S. We can gain improvement by fine-tuning the NN W using labels generated by the teacher without considering their confidence score, i.e. FWL \Σ. This means we just augmented the fine-tuning process by generating a fine-tuning set using teacher which is better than D s in terms of quantity and D w in terms of quality. This baseline is equivalent to setting β = 0. However, we see a big jump in performance when we use FWL to include the estimated label quality from the teacher, leading to the best overall . Sensitivity of the FWL to the Quality of the Weak Annotator. Our proposed setup in FWL requires defining a socalled "weak annotator" to provide a source of weak supervision for unlabelled data. In this section, we study how the quality of the weak annotator may affect the performance of the FWL on the Robust04 dataset. To do so, besides BM25 BID11 ), we use three other weak annotators: vector space model BID12 with binary term occurrence (BTO) weighting schema and vector space model with TF-IDF weighting schema, which are both weaker than BM25, and BM25+RM3 that uses RM3 as the pseudo-relevance feedback method on top of BM25, leading to better labels. FIG2 illustrates the performance of these four weak annotators in terms of their mean average precision (MAP) on the test data, versus the performance of FWL given the corresponding weak annotator. As it is expected, the performance of FWL depends on the quality of the employed weak annotator. The percentage of improvement of FWL over its corresponding weak annotator on the test data is also presented in FIG2. As can be seen, the better the performance of the weak annotator is, the less the improvement of the FWL would be. Training neural networks using large amounts of weakly annotated data is an attractive approach in scenarios where an adequate amount of data with true labels is not available, a situation which often arises in practice. In this paper, we introduced fidelity-weighted learning (FWL), a new student-teacher framework for semi-supervised learning in the presence of weakly labeled data. We applied FWL to document ranking and empirically verified that FWL speeds up the training process and improves over state-of-the-art semi-supervised alternatives. To better understand FWL, we apply FWL to a one-dimensional toy problem to illustrate the various steps. Let ft(x) = sin(x) be the true function (red dotted line in FIG4) from which a small set of observations Ds = {xj,yj} is provided (red points in FIG4). These observation might be noisy, in the same way that labels obtained from a human labeler could be noisy. A weak annotator function fw(x) = 2sinc(x) (magenta line in FIG4) is provided, as an approximation to ft.The task is to obtain a good estimate of ft given the set Ds of strong observations and the weak annotator function fw. We can easily obtain a large set of observations Dw = {xi,ỹi} from fw with almost no cost (magenta points in FIG4).As the teacher, we use standard Gaussian process regression 2 with this kernel: DISPLAYFORM0 where, kRBF(xi,xj) = exp xi −xj 2 2 2 k White (xi,xj) = constant value, ∀x1 = x2 and 0 otherwise We fit only one GP on all the data points (i.e. no clustering). Also during fine-tuning, we set β = 1. The student is a simple feed-forward network with the depth of 3 layers and width of 128 neurons per layer. We have used tanh as the nonlinearity for the intermediate layers and a linear output layer. As the optimizer, we used Adam and the initial learning rate has been set to 0.001. We randomly sample 100 data points from the weak annotator and 10 data points from the true function. We introduce a small amount of noise to the observation of the true function to model the noise in the human labeled data. We consider two experiments:1. A neural network trained on weak data and then fine-tuned on strong data from the true function, which is the most common semi-supervised approach FIG4 ). 2. A teacher-student framework working by the proposed FWL approach. As can be seen in FIG4, FWL by taking into account label confidence, gives a better approximation of the true hidden function. We repeated the above experiment 10 times. The average RMSE with respect to the true function on a set of test points over those 10 experiments for the student, were as follows: Algorithm 1 Clustered Gaussian processes.1: Let N be the sample size, n the sample size of each cluster, K the number of clusters, and ci the center of cluster i. 2: Run K-means with K clusters over all samples with true labels Ds = {xi,yi}.K-means(xi) → c1,c2,...,cK where ci represents the center of cluster Ci containing samples D c i s = {xi,1,xi,2,...xi,n}. 3: Assign each of K clusters a Gaussian process and train them in parallel to approximate the label of each sample. DISPLAYFORM1 where GP c i is trained on D c i s containing samples belonging to the cluster ci. Other elements are defined in Section 2 4: Use trained teacher Tc i to evaluate the soft label and uncertainty for samples from Dsw to compute η2(xt) required for step 3 of FWL. We use T as a wrapper for all teachers {Tc i}. We suggest using several GP = {GPc i} to explore the entire data space more effectively. Even though inducing points and stochastic methods make GPs more scalable we still observed poor performance when the entire dataset was modeled by a single GP. Therefore, the reason for using multiple GPs is mainly empirically inspired by BID13 which is explained in the following: We used the Sparse Gaussian Process implemented in GPflow. The algorithm is scalable in the sense that it is not O(N 3) as original GP is. It introduces inducing points in the data space and defines a variational lower bound for the marginal likelihood. The variational bound can now be optimized by stochastic methods which make the algorithm applicable in large datasets. However, the tightness of the bound depends on the location of inducing points which are found through the optimization process. The pseudo-code of the clustered GP is presented in Algorithm 1. When the main issue is computational resources (when the number of inducing points for each GP is large), we can first choose the number n which is the maximum size of the dataset on which our resources allow to train a GP, then find the number of clusters K = N/n accordingly. The rest of the algorithm remains unchanged. The employed student is proposed in BID3. The first layer of the network models function ψ that learns the representation of the input data samples, i.e. (q,d+,d −), and consists of three components: an embedding function ε: V → R m (where V denotes the vocabulary set and m is the number of embedding dimensions), a weighting function ω: V → R, and a compositionality function: (R m,R) n → R m. More formally, the function ψ is defined as: DISPLAYFORM0 where t q i and t d i denote the i th term in query q respectively document d. The embedding function ε maps each term to a dense m-dimensional real value vector, which is learned during the training phase. The weighting function ω assigns a weight to each term in the vocabulary. It has been shown that ω simulates the effect of inverse document frequency (IDF), which is an important feature in information retrieval BID3.The compositionality function projects a set of n embedding-weighting pairs to an m-dimensional representation, independent from the value of n: DISPLAYFORM1 which is in fact the normalized weighted element-wise summation of the terms' embedding vectors. Again, it has been shown that having global term weighting function along with embedding function improves the performance of ranking as it simulates the effect of inverse document frequency (IDF). In our experiments, we initialize the embedding function ε with word2vec embeddings BID6 pre-trained on Google News and the weighting function ω with IDF.The representation layer is followed by a simple fully connected feed-forward network with l hidden layers followed by a softmax which receives the vector representation of the inputs processed by the representation learning layer and outputs a predictionỹ. Each hidden layer z k in this network computes DISPLAYFORM2 where W k and b k denote the weight matrix and the bias term corresponding to the k th hidden layer and α is the non-linearity. These layers follow a sigmoid output. We employ the cross entropy loss: DISPLAYFORM3 where B is a batch of data samples. We use Gaussian Process as the teacher and pass the mean of GP through the same function g that is applied on the output of the student network. h is an aggregation function that takes variance over several dimensions and outputs a single measure of variance. As a reasonable choice, the aggregating function h in our sentiment classification task (three classes) is mean of variances over dimensions. In the teacher, linear combinations of different kernels are used in our experiments. We use sparse variational GP regression 3 with this kernel: DISPLAYFORM0 where, DISPLAYFORM1 +xi.xj k White (xi,xj) = constant value, ∀x1 = x2 and 0 otherwise We empirically found l = 1 satisfying value for the length scale of Matern3/2 kernels. We also set σ0 = 0 to obtain a homogeneous linear kernel. The constant value of K W hite (.,.) determines the level of noise in the labels. This is different from the noise in weak labels. This term explains the fact that even in true labels there might be a trace of noise due to the inaccuracy of human labelers. We set the number of clusters in the clustered GP algorithm for the ranking task to 50. The weak annotator is BM25 BID11 ), a well-known unsupervised retrieval method. This method heuristically scores a given pair of query-document based on the statistics of their matched terms. In the pairwise document ranking setup,ỹi for a given sample xj = (q,d DISPLAYFORM0 is the score obtained from the weak annotator. Collections We use two standard TREC collections for the task of ad-hoc retrieval: The first collection (Robust04) consists of 500k news articles from different news agencies as a homogeneous collection. The second collection (ClueWeb) is ClueWeb09 Category B, a large-scale web collection with over 50 million English documents, which is considered as a heterogeneous collection. Spam documents were filtered out using the Waterloo spam scorer 4 BID1 with the default threshold 70%.Data with true labels We take query sets that contain human-labeled judgments: a set of 250 queries (TREC topics 301-450 and 601-700) for the Robust04 collection and a set of 200 queries (topics for the experiments on the ClueWeb collection. For each query, we take all documents judged as relevant plus the same number of documents judged as non-relevant and form pairwise combinations among them. Data with weak labels We create a query set Q using the unique queries appearing in the AOL query logs BID9 . This query set contains web queries initiated by real users in the AOL search engine that were sampled from a three-month period from March 2006 to May 2006. We applied standard pre-processing BID3 a) on the queries: We filtered out a large volume of navigational queries containing URL substrings ("http", "www.", ".com", ".net", ".org", ".edu"). We also removed all non-alphanumeric characters from the queries. For each dataset, we took queries that have at least ten hits in the target corpus using our weak annotator method. Applying all these steps, We collect 6.15 million queries to train on in Robust04 and 6.87 million queries for ClueWeb. To prepare the weakly labeled training set Dw, we take the top 1,000 retrieved documents using BM25 for each query from training query set Q, which in total leads to ∼ |Q|×10 6 training samples. Setup For the evaluation of the whole model, we conducted 3-fold cross-validation. However, for each dataset, we first tuned all the hyper-parameters of the student in the first step on the set with true labels using batched GP bandits with an expected improvement acquisition function BID4 and kept the optimal parameters of the student fixed for all the other experiments. The size and number of hidden layers for the student is selected from {64,128,256,512}. The initial learning rate and the dropout parameter were selected from {10 −3,10 −5} and {0.0,0.2,0.5}, respectively. We considered embedding sizes of {300,500}. The batch size in our experiments was set to 128. We use ReLU BID7 as a non-linear activation function α in student. We use the Adam optimizer for training, and dropout BID14 as a regularization technique. At inference time, for each query, we take the top 2,000 retrieved documents using BM25 as candidate documents and re-rank them using the trained models. We use the Indri 5 implementation of BM25 with default parameters (i.e., k1 = 1.2, b = 0.75, and k3 = 1,000).

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We propose Fidelity-weighted Learning, a semi-supervised teacher-student approach for training neural networks using weakly-labeled data.

A backward model of previous (state, action) given the next state, i.e. P(s_t, a_t | s_{t+1}), can be used to simulate additional trajectories terminating at states of interest! Improves RL learning efficiency.