Question: Represent the scientific article to find a short TLDR

Question: The attention layer in a neural network model provides insights into the model’s reasoning behind its prediction, which are usually criticized for being opaque. Recently, seemingly contradictory viewpoints have emerged about the interpretability of attention weights . Amid such confusion arises the need to understand attention mechanism more systematically. In this work, we attempt to fill this gap by giving a comprehensive explanation which justifies both kinds of observations (i.e., when is attention interpretable and when it is not). Through a series of experiments on diverse NLP tasks, we validate our observations and reinforce our claim of interpretability of attention through manual evaluation. Attention is a way of obtaining a weighted sum of the vector representations of a layer in a neural network model . It is used in diverse tasks ranging from machine translation , language modeling to image captioning , and object recognition . Apart from substantial performance benefit , attention also provides interpretability to neural models (; ;) which are usually criticized for being black-box function approximators . There has been substantial work on understanding attention in neural network models. On the one hand, there is work on showing that attention weights are not interpretable, and altering them does not significantly affect the prediction . While on the other hand, some studies have discovered how attention in neural models captures several linguistic notions of syntax and coreference (; ;). Amid such contrasting views arises a need to understand the attention mechanism more systematically. In this paper, we attempt to fill this gap by giving a comprehensive explanation which justifies both kinds of observations. The of; have been mostly based on text classification experiments which might not generalize to several other NLP tasks. In Figure 1, we report the performance on text classification, Natural Language Inference (NLI) and Neural Machine Translation (NMT) of two models: one trained with neural attention and the other trained with attention weights fixed to a uniform distribution. The show that the attention mechanism in text classification does not have an impact on the performance, thus, making inferences about interpretability of attention in these models might not be accurate. However, on tasks such as NLI and NMT uniform attention weights degrades the performance substantially, indicating that attention is a crucial component of the model for these tasks and hence the analysis of attention's interpretability here is more reasonable. In comparison to the existing work on interpretability, we analyze attention mechanism on a more diverse set of NLP tasks that include text classification, pairwise text classification (such as NLI), and text generation tasks like neural machine translation (NMT). Moreover, we do not restrict ourselves to a single attention mechanism and also explore models with self-attention. For examining the interpretability of attention weights, we perform manual evaluation. Our key contributions are: 1. We extend the analysis of attention mechanism in prior work to diverse NLP tasks and provide a comprehensive picture which alleviates seemingly contradicting observations. 2. We identify the conditions when attention weights are interpretable and correlate with feature importance measures -when they are computed using two vectors which are both functions of the input (Figure 1b, c). We also explain why attention weights are not interpretable when the input has only single sequence (Figure 1a), an observation made by , by showing that they can be viewed as a gating unit. 3. We validate our hypothesis of interpretability of attention through manual evaluation. We investigate the attention mechanism on the following three task categories. 1. Single Sequence tasks are those where the input consists of a single text sequence. For instance, in sentiment analysis, the task is to classify a review as positive or negative. This also includes other text classification tasks such as topic categorization. For the experiments, in this paper, we use three review rating datasets: Stanford Sentiment Treebank , IMDB and and Multi-Genre Natural Language Inference (MultiNLI) datasets for our analysis. For question answering, similar to , we use CNN News Articles and three tasks of the original babI dataset in our experiments, i.e., using one, two and three supporting statements as the context for answering the questions. 3. Generation tasks involve generating a sequence based on the input sequence. Neural Machine translation is an instance of generation task which comprises of translating a source text to a target language given translation pairs from a parallel corpus. For our experiments, we use three English-German datasets: Multi30k , En-De News Commentary v11 from WMT16 translation task 3 and full En-De WMT13 dataset. In this section, we give a brief overview of the neural attention-based models we analyze for different categories of tasks listed in Section 2. The overall architecture for each category is shown in Fig 1. For single sequence tasks, we adopt the model architecture from;. For a given input sequence x ∈ R T ×|V |, where T and |V | are the number of tokens and vocabulary size, we first represent each token with its d-dimensional GloVe embedding to obtain x e ∈ R T ×d. Next, we use a Bi-RNN encoder (Enc) to obtain an m-dimensional contextualized representation of tokens: h = Enc(x e) ∈ R T ×m. Then, we use the additive formulation of attention proposed by for computing attention weights α i for all tokens defined as: where are the parameters of the model. Finally, the weighted instance representation: m is fed to a dense layer (Dec) followed by softmax to obtain predictionŷ = σ(Dec(h α)) ∈ R |Y|, where |Y| denotes the label set size. We also analyze the hierarchical attention model , which involves first computing attention over the tokens to obtain a sentence representation. This is followed by attention over sentences to obtain an instance representation h α, which is fed to a dense layer for obtaining prediction (ŷ). At both word and sentence level the attention is computed similar to as defined in Equation 1. For pair sequence, the input consists of two text sequences: x ∈ R T1×|V |, y ∈ R T2×|V | of length T 1 and T 2. In NLI, x indicates premise and y is hypothesis while in question answering, it is the question and paragraph respectively. , we use two separate RNNs for encoding both the sequences to obtain {h where similar to Equation 1, W 1, W 2 ∈ R d×d denotes the projection matrices and c ∈ R d is a parameter vector. Finally, the representation obtained from a weighted sum of tokens in x: is fed to a classifier for prediction. We also explore a variant of the above attention proposed by Rocktäschel et al.. Instead of keeping the RNN encoders of both the sequences independent, Rocktäschel et al. use conditional encoding where the encoder of y is initialized with the final state of x's encoder. This allows the model to obtain a conditional encoding {h y 1, ..., h y T2} of y given the sequence x. Moreover, unlike the previous model, attention over the tokens of x is defined as follows: where ], e T1 ∈ R T1 is a vector of ones and outer product W 2 h y T2 ⊗ e T1 denotes repeating linearly transformed h y T2 as many times as words in the sequence x (i.e. T 1 times). In this paper, for generation tasks, we focus on Neural Machine Translation (NMT) problem which involves translating a given source text sentence x ∈ R T1×|V1| to a sequence y ∈ R T2×|V2| in the target language. The model comprises of two components: (a) an encoder which computes a representation for each source sentence and (b) a decoder which generates a target word at each time step. In this work, we utilize RNN based encoder and decoder models. For each input sentence x, we first obtain a contextualized representation {h 1, ..., h T1} of its tokens using a multi-layer Bi-RNN. Then, at each time step t, the decoder has a hidden state defined as In our work, we compute α t,i as proposed by and. The former computes attention weights using a feed-forward network, i.e., α t,i = w T tanh(W [c t ; h i]) while the latter define it simply as α t,i = c T t h i. We also examine self-attention based models on all three categories of tasks. For single and pair sequence tasks, we fine-tune pre-trained BERT model on the downstream task. In pair sequence tasks, instead of independently encoding each text, we concatenate both separated by a delimiter and pass it to BERT model. Finally, the embedding corresponding to [CLS] token is fed to a feed-forward network for prediction. For neural machine translation, we use Transformer model proposed by with base configuration. In this section, we attempt to address the question: Is attention an explanation? through a series of experiments which involve analyzing attention weights in a variety of models (§3) on multiple tasks (§2). , we take the definition of explainability of attention as: inputs with high attention weights are responsible for model output.; have extensively investigated this aspect for certain class of problems and have shown that attention does not provide an explanation. However, another series of work (; ;) has shown that attention does encode several linguistic notions. In our work, we claim that the findings of both the line of work are consistent. We note that the observations of the former works can be explained based on the following proposition. Proposition 4.1. Attention mechanism as defined in Equation 1 as for single sequence tasks can be interpreted as a gating unit in the network. Proof: The attention weighted averaging computed in Equation 1 for single sequence tasks can be interpreted as gating proposed by which is defined as where x ∈ R m is the input and × denotes product between transformed input f (x) ∈ R m and its computed gating scores σ(g(x)) ∈ R. Equation 1 can be reduced to the above form by taking f as an identity function and defining g(x) = c T tanh(W x + b) ∈ R and replacing σ with softmax. We note that the same reduction does not hold in the case of pair sequence and generation tasks where attention along with input also depends on another text sequence Y and current hidden state c t, respectively. Thus, attention mechanism for these tasks take the form h(x, y) = f (x) × σ(g(x, y)), which does not reduce to the above equation for gating unit. Table 1: Evaluation on single sequence tasks. We report the base performance of attention models and absolute change in accuracy for its variant. We note that across all datasets, degradation in performance on altering attention weights during inference is more compared to varying them during both training and inference. Overall, the change in performance is less compared to other tasks. Please refer to §4.1 for more details. Based on the above proposition, we argue that weights learned in single sequence tasks cannot be interpreted as attention, and therefore, they do not reflect the reasoning behind the model's prediction. This justifies the observation that for the single sequence tasks examined in; , attention weights do not correlate with feature importance measures and permuting them does not change the prediction of the model. In light of this observation, we revisit the explainability of attention weights by asking the following questions. In this section, we compare the performance of various attention mechanism described in §3 for different categories of tasks listed in §2. For each model, we analyze its three variants defined as: • Uniform denotes the case when all the inputs are given equal weights, i.e., α i = 1/T, ∀i ∈ {1, ..., T}. This is similar to the analysis performed by. However, we consider two scenarios when the weights are kept fixed both during training and inference (Train+Infer) and only during inference (Infer). • Random refers to the variant where all the weights are randomly sampled from a uniform distribution: α i ∼ U, ∀i ∈ {1, ..., T}, this is followed by normalization. Similar to Uniform, we analyze both Train+Infer and Infer. • Permute refers to the case when the learned attention weights are randomly permuted during inference, i.e., α = shuffle(α). Unlike the previous two, here we restrict our analysis to only permuting during inference as Tensorflow currently does not support backpropagation with shuffle operation. Effect on single sequence tasks: The evaluation on single sequence datasets: SST, IMDB, AG News, and YELP are presented in Table 1. We observe that Train+Infer case of Uniform and Random attentions gives around 0.5 and 0.9 average decrease in accuracy compared to the base model. However, in Infer scenario the degradation on average increases to 3.9 and 4.5 absolute points respectively. This is so because the model becomes more robust to handle altered weights in the former case. The reduction in performance from Permute comes around to 4.2 across all datasets and models. The support the observation of; that alternating attention in text classification task does not have much effect on the model output. The slight decrease in performance can be attributed to corrupting the existing gating mechanism which has been shown to give some improvement (; ;). Effect on pair sequence and generation tasks: The on pair sequence and generation tasks are summarized in Table 2 and 3, respectively. Overall, we find that the degradation in performance from altering attention weights in case of pair sequence and generation tasks is much more substantial than single sequence tasks. For instance, in Uniform (Train+Infer), the average relative decrease Table 2: The performance comparison of attention based models and their variants on pair sequence tasks. We find that the degradation in performance is much more than single sequence tasks. Table 3: Evaluation on neural machine translation. Similar to pair sequence tasks, we find that the deterioration in performance is much more substantial than single sequence tasks. Please refer to §4.1 for more details. in performance of single sequence tasks is 0.1% whereas in case of pair sequence and generation tasks it is 49.5% and 51.2% respectively. The thereby validate our Proposition 4.1 and show that altering attention does affect model output for a task where the attention layer cannot be modeled as a gating unit in the network. Visualizing the effect of permuting attention weights: To further reinforce our claim, similar to , we report the median of Total Variation Distance (TVD) between new and original prediction on permuting attention weights for each task. The TVD between two predictionŝ y 1 andŷ 2 is defined as: where |Y| denotes the total number of classes in the problem. We use TVD for measuring the change in output distribution on permuting the attention weights. In Figure 2, we report the relationship between the maximum attention value and the median induced change in model output over 100 permutations on all categories of tasks. For NMT task, we present change in output at the 25th-percentile length of sentences for both datasets. Overall, we find that for single sequence tasks even with the maximum attention weight in range [0.75, 1.0], the change in prediction is considerably small (the violin plots are to the left of the figure) compared to the pair sequence and generation tasks (the violin plots are to the right of the figure). In this section, similar to the analysis of , we investigate the importance of attention weights only when one weight is removed. Let i * be the input corresponding to the highest attention weights and let r be any randomly selected input. We denote the original model's prediction as p and output after removing i * and r input as q {i *} and q {r} respectively. Now, to measure the impact of removing i * relative to any randomly chosen input r on the model output, we compute the difference of Jensen-Shannon (JS) divergence between JS(p, q {i *} ) and JS(p, q {r} ) given as: ∆JS = JS(p, q {i *} ) − JS(p, q {r} ). The relationship between the difference of attention weights corresponding to i * and r, i.e., α i * −α r and ∆JS for different tasks is presented in Figure 3. In general, we found that for single sequence tasks, the change JS divergence is small even for cases when the difference in attention weight is considerable. However, for pair sequence and generation tasks, there is a substantial change in the model output. In this section, we analyze the importance of attention weights on the performance of self-attention based models as described in §3.4. We report the accuracy on single, and pair sequence tasks and BLEU score for NMT on WMT13 dataset on permuting the attention weights of layers cumulatively. For Transformer model, we analyze the effect of altering attention weights in encoder, decoder, and across encoder-decoder (denoted by Across). The are presented in Figure 4. Overall, we find that unlike the pattern observed in §4.1 and §4.2 for single sequence tasks, altering weights in self-attention based models does have a substantial effect on the performance. We note that this is because while computing attention weights over all tokens with respect to a given token, Proposition 4.1 does not hold. Thus, altering them does have an impact across all three tasks. We note that in the case of transformer model, altering the weights in the first step of Decoder and in Across has maximum effect as it almost stops the flow of information from encoder to decoder. Figure 5: Manual evaluation of interpretability of attention weights on single and pair sequence tasks. Although with original weights the attention does remain interpretable on both tasks but in the case of single sequence tasks making it meaningless does not change the prediction substantially. However, the same does not hold with pair sequence tasks. To determine if attention weights are human interpretable, here, we address the question of interpretability of attention weights by manually analyzing them on a representative dataset of single and pair sequence task. For each task, we randomly sample 100 samples with original attention weights and 100 with randomly permuted weights. Then, we shuffle all 200 samples together and present them to annotators for deciding whether the top three highest weighted words are relevant for the model's prediction. The overall are reported in Figure 5. Cohen's kappa score of inter-annotator agreement on IMDB and babI is 0.84 and 0.82, respectively, which shows near-perfect agreement . We find that in both single and pair sequence tasks, the attention weights in samples with original weights do make sense in general (highlighted with blue color). However, in the former case, the attention mechanism learns to give higher weights to tokens relevant to both kinds of sentiment. For instance, in "This is a great movie. Too bad it is not available on home video.", tokens great, too, and bad get the highest weight. Such examples demonstrate that the attention mechanism in single sequence tasks works like a gating unit, as shown in §4.1. For permuted samples, in the case of single sequence, the prediction remains correct in majority although the attention weights were meaningless. For example, in "This movie was terrible. the acting was lame, but it's hard to tell since the writing was so bad.", the prediction remains the same on changing attention weights from underlined to bold tokens. However, this does not hold with the pair sequence task. This shows that attention weights in single sequence tasks do not provide a reason for the prediction, which in the case of pairwise tasks, attention do reflect the reasoning behind model output. In this paper, we addressed the seemingly contradictory viewpoint over explainability of attention weights in NLP. On the one hand, some works have demonstrated that attention weights are not interpretable, and altering them does not affect the model output while several others have shown that attention captures several linguistic notions in the model. We extend the analysis of prior works to diverse NLP tasks and demonstrate that attention weights are interpretable and are correlated with feature importance measures. However, this holds only for cases when attention weights are essential for model's prediction and cannot simply be reduced to a gating unit. Through a battery of experiments, we validate our claims and reinforce them through manual evaluation.

Analysis of attention mechanism across diverse NLP tasks.

Represent the short summary for finding the corresponding scientific article (long)

Language generation using seq2seq models which produce word embeddings instead of a softmax based distribution over the vocabulary at each step enabling much faster training while maintaining generation quality


Question: Represent the scientific article to find a short TLDR

Question: Generative models forsource code are an interesting structured prediction problem, requiring to reason about both hard syntactic and semantic constraints as well as about natural, likely programs. We present a novel model for this problem that uses a graph to represent the intermediate state of the generated output. Our model generates code by interleaving grammar-driven expansion steps with graph augmentation and neural message passing steps. An experimental evaluation shows that our new model can generate semantically meaningful expressions, outperforming a range of strong baselines. Learning to understand and generate programs is an important building block for procedural artificial intelligence and more intelligent software engineering tools. It is also an interesting task in the research of structured prediction methods: while imbued with formal semantics and strict syntactic rules, natural source code carries aspects of natural languages, since it acts as a means of communicating intent among developers. Early works in the area have shown that approaches from natural language processing can be applied successfully to source code BID11, whereas the programming languages community has had successes in focusing exclusively on formal semantics. More recently, methods handling both modalities (i.e., the formal and natural language aspects) have shown successes on important software engineering tasks BID22 BID4 BID1 and semantic parsing (; BID20).However, current generative models of source code mostly focus on only one of these modalities at a time. For example, program synthesis tools based on enumeration and deduction BID24 BID19 BID8 BID7 are successful at generating programs that satisfy some (usually incomplete) formal specification but are often obviously wrong on manual inspection, as they cannot distinguish unlikely from likely, "natural" programs. On the other hand, learned code models have succeeded in generating realistic-looking programs BID17 BID5 BID18 BID20 ). However, these programs often fail to be semantically relevant, for example because variables are not used consistently. In this work, we try to overcome these challenges for generative code models and present a general method for generative models that can incorporate structured information that is deterministically available at generation time. We focus our attention on generating source code and follow the ideas of program graphs BID1 ) that have been shown to learn semantically meaningful representations of (pre-existing) programs. To achieve this, we lift grammar-based tree decoder models into the graph setting, where the diverse relationships between various elements of the generated code can be modeled. For this, the syntax tree under generation is augmented with additional edges denoting known relationships (e.g., last use of variables). We then interleave the steps of the generative procedure with neural message passing BID9 to compute more precise representations of the intermediate states of the program generation. This is fundamentally different from sequential generative models of graphs BID14 BID23, which aim to generate all edges and nodes, whereas our graphs are deterministic augmentations of generated trees. To summarize, we present a) a general graph-based generative procedure for highly structured objects, incorporating rich structural information; b) ExprGen, a new code generation task focused on (a, u) ← insertChild(a,) if is nonterminal type then 6:a ← Expand(c, a, u) 7: return a int ilOffsetIdx = Array. IndexOf(sortedILOffsets, map. ILOffset); int nextILOffsetIdx = ilOffsetIdx + 1; int nextMapILOffset = nextILOffsetIdx < sortedILOffsets. Length? sortedILOffsets [nextILOffsetIdx]: int. MaxValue; Figure 1: Example for ExprGen, target expression to be generated is marked. Taken from BenchmarkDotNet, lightly edited for formatting.generating small, but semantically complex expressions conditioned on source code context; and c) a comprehensive experimental evaluation of our generative procedure and a range of baseline methods from the literature. The most general form of the code generation task is to produce a (partial) program in a programming language given some context information c. This context information can be natural language (as in, e.g., semantic parsing), input-output examples (e.g., inductive program synthesis), partial program sketches, etc. Early methods generate source code as a sequence of tokens BID11 BID10 and sometimes fail to produce syntactically correct code. More recent models are sidestepping this issue by using the target language's grammar to generate abstract syntax trees (ASTs) BID17 BID5 BID18; BID20, which are syntactically correct by construction. In this work, we follow the AST generation approach. The key idea is to construct the AST a sequentially, by expanding one node at a time using production rules from the underlying programming language grammar. This simplifies the code generation task to a sequence of classification problems, in which an appropriate production rule has to be chosen based on the context information and the partial AST generated so far. In this work, we simplify the problem further -similar to BID17; BID5 -by fixing the order of the sequence to always expand the left-most, bottom-most nonterminal node. Alg. 1 illustrates the common structure of AST-generating models. Then, the probability of generating a given AST a given some context c is DISPLAYFORM0 where a t is the production choice at step t and a <t the partial syntax tree generated before step t. Code Generation as Hole Completion We introduce the ExprGen task of filling in code within a hole of an otherwise existing program. This is similar, but not identical to the auto-completion function in a code editor, as we assume information about the following code as well and aim to generate whole expressions rather than single tokens. The ExprGen task also resembles program sketching but we give no other (formal) specification other than the surrounding code. Concretely, we restrict ourselves to expressions that have Boolean, arithmetic or string type, or arrays of such types, excluding expressions of other types or expressions that use project-specific APIs. An example is shown in Fig. 1. We picked this subset because it already has rich semantics that can require reasoning about the interplay of different variables, while it still only relies on few operators and does not require to solve the problem of open vocabularies of full programs, where an unbounded number of methods would need to be considered. In our setting, the context c is the pre-existing code around a hole for which we want to generate an expression. This also includes the set of variables v 1,..., v that are in scope at this point, which can be used to guide the decoding procedure BID17. Note, however, that our method is not restricted to code generation and can be easily extended to all other tasks and domains that can be captured by variations of Alg. 1 (e.g. in NLP). To tackle the code generation task presented in the previous section, we have to make two design choices: (a) we need to find a way to encode the code context c, v 1,..., v and (b) we need to construct a model that can learn p(a t | c, a <t) well. We do not investigate the question of encoding the context in this paper, and use two existing methods in our experiments in Sect. 5. Both these encoders yield a distributed vector representation for the overall context, representations h t1,..., h t T for all tokens in the context, and separate representations for each of the in-scope variables v 1,..., v, summarizing how each variable is used in the context. This information can then be used in the generation process, which is the main contribution of our work and is described in this section. Overview Our decoder model follows the grammar-driven AST generation strategy of prior work as shown in Alg. 1. The core difference is in how we compute the representation of the node to expand. BID17 construct it entirely from the representation of its parent in the AST using a log-bilinear model. BID20 construct the representation of a node using the parents of the AST node but also found it helpful to take the relationship to the parent node (e.g. "condition of a while") into account. on the other hand propose to take the last expansion step into account, which may have finished a subtree "to the left". In practice, these additional relationships are usually encoded by using gated recurrent units with varying input sizes. We propose to generalize and unify these ideas using a graph to structure the flow of information in the model. Concretely, we use a variation of attribute grammars BID12 from compiler theory to derive the structure of this graph. We associate each node in the AST with two fresh nodes representing inherited resp. synthesized information (or attributes). Inherited information is derived from the context and parts of the AST that are already generated, whereas synthesized information can be viewed as a "summary" of a subtree. In classical compiler theory, inherited attributes usually contain information such as declared variables and their types (to allow the compiler to check that only declared variables are used), whereas synthesized attributes carry information about a subtree "to the right" (e.g., which variables have been declared). Traditionally, to implement this, the language grammar has to be extended with explicit rules for deriving and synthesizing attributes. To transfer this idea to the deep learning domain, we represent attributes by distributed vector representations and train neural networks to learn how to compute attributes. Our method for getRepresentation from Alg. 1 thus factors into two parts: a deterministic procedure that turns a partial AST a <t into a graph by adding additional edges that encode attribute relationships, and a graph neural network that learns from this graph. Notation Formally, we represent programs as graphs where nodes u, v,... are either the AST nodes or their associated attribute nodes, and typed directed edges u, τ, v ∈ E connect the nodes according to the flow of information in the model. The edge types τ represent different syntactic or semantic relations in the information flow, discussed in detail below. We write E v for the set of incoming edges into v. We also use functions like parent(a, v) and lastSibling(a, v) that look up and return nodes from the AST a (e.g. resp. the parent node of v or the preceding AST sibling of v). if v is variable then 7: DISPLAYFORM0 Example Consider the AST of the expression i -j shown in FIG0 (annotated with attribute relationships) constructed step by step by our model. The AST derivation using the programming language grammar is indicated by shaded s, nonterminal nodes are shown as rounded rectangles, and terminal nodes are shown as rectangles. We additionally show the variables given within the context as dashed rectangles at the bottom. First, the root node, Expr, was expanded using the production rule: Expr =⇒ Expr -Expr. Then, its two nonterminal children were in turn expanded to the set of known variables using the produc-Published as a conference paper at ICLR 2019 Attribute nodes are shown overlaying their corresponding AST nodes. For example, the root node is associated with its inherited attributes node 0 and with node 10 for its synthesized attributes. For simplicity, we use the same representation for inherited and synthesized attributes of terminal nodes. Edges in a <t We discuss the edges used in our neural attribute grammars (NAG) on our example below, and show them in FIG0 using different edge drawing styles for different edge types. Once a node is generated, the edges connecting this node can be deterministically added to a <t (precisely defined in Alg. 2). The list of different edge types used in our model is as follows:• Child (red) edges connect an inherited attribute node to the inherited attributes nodes of its children, as seen in the edges from node 0. These are the connections in standard syntaxdriven decoders BID17 BID18; BID20 ).• Parent (green) edges connect a synthesized attribute node to the synthesized attribute node of its AST parent, as seen in the edges leading to node 10. These are the additional connections used by the R3NN decoder introduced by BID18.• NextSib (black) edges connect the synthesized attribute node to the inherited attribute node of its next sibling (e.g. from node 5 to node 6). These allow information about the synthesized attribute nodes from a fully generated subtree to flow to the next subtree.• NextUse (orange) edges connect the attribute nodes of a variable (since variables are always terminal nodes, we do not distinguish inherited from synthesized attributes) to their next use. Unlike BID1, we do not perform a dataflow analysis, but instead just follow the lexical order. This can create edges from nodes of variables in the context c (for example, from node 1 to 4 in FIG0), or can connect AST leaf nodes that represent multiple uses of the same variable within the generated expressions.• NextToken (blue) edges connect a terminal node (a token) to the next token in the program text, for example between nodes 4 and 6. • InhToSyn edges (not shown in FIG0) connect the inherited attributes nodes to its synthesized attribute nodes. This is not strictly adding any information, but we found it to help with training. The panels of FIG0 show the timesteps at which the representations of particular attribute nodes are computed and added to the graph. For example, in the second step, the attributes for the terminal token i (node 4) in FIG0 are computed from the inherited attributes of its AST parent Expr (node 3), the attributes of the last use of the variable i (node 1), and the node label i. In the third step, this computed attribute is used to compute the synthesized attributes of its AST parent Expr (node 5).Attribute Node Representations To compute the neural attribute representation h v of an attribute node v whose corresponding AST node is labeled with v, we first obtain its incoming edges using Alg. 2 and then use the state update function from Gated Graph Neural Networks (GGNN) BID13. Thus, we take the attribute representations h ui at edge sources u i, transform them according to the corresponding edge type t i using a learned function f ti, aggregate them (by elementwise summation) and combine them with the learned embedding emb(v) of the node label v using a function g: DISPLAYFORM1 In practice, we use a single linear layer for f ti and implement g as a gated recurrent unit. We compute node representations in such an order that all h ui appearing on the right of are already computed. This is possible as the graphs obtained by repeated application of Alg. 2 are directed acyclic graphs rooted in the inherited attribute node of the root node of the AST. We initialize the representation of the root inherited attribute to the representation returned by the encoder for the context information. Choosing Productions, Variables & Literals We can treat picking production rules as a simple classification problem over all valid production rules, masking out those choices that do not correspond to the currently considered nonterminal. For a nonterminal node v with label v and inherited attributes h v, we thus define DISPLAYFORM2 Here, m v is a mask vector whose value is 0 for valid productions v ⇒... and −∞ for all other productions. In practice, we implement e using a linear layer. Similarly, we pick variables from the set of variables V in scope using their representations h vvar (initially the representation obtained from the context, and later the attribute representation of the last node in the graph in which they have been used) by using a pointer network BID25. Concretely, to pick a variable at node v, we use learnable linear function k and define DISPLAYFORM3 Note that since the model always picks a variable from the set of in-scope variables V, this generation model can never predict an unknown or out-of-scope variable. Finally, to generate literals, we combine a small vocabulary L of common literals observed in the training data and special UNK tokens for each type of literal with another pointer network that can copy one of the tokens t 1... t T from the context. Thus, to pick a literal at node v, we define DISPLAYFORM4 Note that this is the only operation that may produce an unknown token (i.e. an UNK literal). In practice, we implement this by learning two functions s L and s c, such that s L (h v) produces a score for each token from the vocabulary and s c (h v, h ti) computes a score for copying token t i from the context. By computing a softmax over all ing values and normalizing it by summing up entries corresponding to the same constant, we can learn to approximate the desired P (lit | h v). The different shapes and sizes of generated expressions complicate an efficient training regime. However, note that given a ground truth target tree, we can easily augment it with all additional edges according to Alg. 2. Given that full graph, we can compute a propagation schedule (intuitively, a topological ordering of the nodes in the graph, starting in the root node) that allows to repeatedly apply to obtain representations for all nodes in the graph. By representing a batch of graphs as one large (sparse) graph with many disconnected components, similar to BID1, we can train our graph neural network efficiently. We have released the code for this on https://github.com/Microsoft/graph-based-code-modelling.Our training procedure thus combines an encoder (cf. Sect. 5), whose output is used to initialize the representation of the root and context variable nodes in our augmented syntax graph, the sequential graph propagation procedure described above, and the decoder choice functions and. We train the system end-to-end using a maximum likelihood objective without pre-trained components. Additional Improvements We extend with an attention mechanism BID16 that uses the state h v of the currently expanded node v as a key and the context token representations h t1,..., h t T as memories. Experimentally, we found that extending Eqs. 4, 5 similarly did not improve , probably due to the fact that they already are highly dependent on the context information. Following BID20, we provide additional information for Child edges. To allow this, we change our setup so that some edge types also require an additional label, which is used when computing the messages sent between different nodes in the graph. Concretely, we extend by considering sets of unlabeled edges E v and labeled edges E v: DISPLAYFORM0 Thus for labeled edge types, f ti takes two inputs and we additionally introduce a learnable embedding for the edge labels. In our experiments, we found it useful to label Child with tuples consisting of the chosen production and the index of the child, i.e., in FIG0, we would label the edge from 0 to 3 with, the edge from 0 to 6 with, etc. Furthermore, we have extended pickProduction to also take the information about available variables into account. Intuitively, this is useful in cases of productions such as Expr =⇒ Expr. Length, which can only be used in a well-typed derivation if an array-typed variable is available. Thus, we extend e(h v) from to additionally take the representation of all variables in scope into account, i.e., e(h v, r({h vvar | var ∈ V})), where we have implemented r as a max pooling operation. Source code generation has been studied in a wide range of different settings BID0. We focus on the most closely related works in language modeling here. Early works approach the task by generating code as sequences of tokens BID11 BID10, whereas newer methods have focused on leveraging the known target grammar and generate code as trees BID17 BID5 BID18; BID20 ) (cf. Sect. 2 for an overview). While modern models succeed at generating "natural-looking" programs, they often fail to respect simple semantic rules. For example, variables are often used without initialization or written several times without being read inbetween. Existing tree-based generative models primarily differ in what information they use to decide which expansion rule to use next. BID17 consider the representation of the immediate parent node, and suggest to consider more information (e.g., nearby tokens). BID18 compute a fresh representation of the partial tree at each expansion step using R3NNs (which intuitively perform a leaf-to-root traversal followed by root-to-leaf traversal of the AST). The PHOG model BID5 conditions generation steps on the of learned (decision tree-style) programs, which can do bounded AST traversals to consider nearby tokens and non-terminal nodes. The language also supports a jump to the last node with the same identifier, which can serve as syntactic approximation of data-flow analysis. BID20 only use information about the parent node, but use neural networks specialized to different non-terminals to gain more fine-grained control about the flow of information to different successor nodes. Finally, BID2 and follow a left-to-right, depth-first expansion strategy, but thread updates to single state (via a gated recurrent unit) through the overall generation procedure, thus giving the pickProduction procedure access to the full generation history as well as the representation of the parent node. BID2 also suggest the use of attribute grammars, but use them to define a deterministic procedure that collects information throughout the generation process, which is provided as additional feature. As far as we are aware, previous work has not considered a task in which a generative model fills a hole in a program with an expression. Lanuage model-like methods take into account only the lexicographically previous context of code. The task of BID21 is near to our ExprGen, but instead focuses on filling holes in sequences of API calls. There, the core problem is identifying the correct function to call from a potentially large set of functions, given a sequence context. In contrast, ExprGen requires to handle arbitrary code in the context, and then to build possibly complex expressions from a small set of operators. BID1 consider similar context, but are only picking a single variable from a set of candidates, and thus require no generative modeling. Dataset We have collected a dataset for our ExprGen task from 593 highly-starred open-source C # projects on GitHub, removing any near-duplicate files, following the work of BID15. We parsed all C # files and identified all expressions of the fragment that we are considering (i.e., restricted to numeric, Boolean and string types, or arrays of such values; and not using any user-defined functions). We then remove the expression, perform a static analysis to determine the necessary context information and extract a sample. For each sample, we create an abstract syntax tree by coarsening the syntax tree generated by the C # compiler Roslyn. This ed in 343 974 samples overall with 4.3 (±3.8) tokens per expression to generate, or alternatively 3.7 (± 3.1) production steps. We split the data into four separate sets. A "test-only" dataset is made up from ∼100k samples generated from 114 projects. The remaining data we split into training-validation-test sets (3 : 1 : 1), keeping all expressions collected from a single source file within a single fold. Samples from our dataset can be found in the supplementary material. Our decoder uses the grammar made up by 222 production rules observed in the ASTs of the training set, which includes rules such as Expr =⇒ Expr + Expr for binary operations, Expr =⇒ Expr. Equals(Expr) for built-in methods, etc. Encoders We consider two models to encode context information. Seq is a two-layer bi-directional recurrent neural network (using a GRU) to encode the tokens before and after the "hole" in which we want to generate an expression. Additionally, it computes a representation for each variable var in scope in the context in a similar manner: For each variable var it identifies usages before/after the hole and encodes each of them independently using a second bi-directional two-layer GRU, which processes a window of tokens around each variable usage. It then computes a representation for var by average pooling of the final states of these GRU runs. The second encoder G is an implementation of the program graph approach introduced by BID1. We follow the transformation used for the Varmisuse task presented in that paper, i.e., the program is transformed into a graph, and the target expression is replaced by a fresh dummy node. We then run a graph neural network for 8 steps to obtain representations for all nodes in the graph, allowing us to read out a representation for the "hole" (from the introduced dummy node) and for all variables in context. The used context information captured by the GNN is a superset of what existing methods (e.g. language models) consider. Baseline Decoders We compare our model to re-implementations of baselines from the literature. As our ExprGen task is new, re-using existing implementations is hard and problematic in comparison. Most recent baseline methods can be approximated by ablations of our model. We experimented with a simple sequence decoder with attention and copying over the input, but found it to be substantially weaker than other models in all regards. Next, we consider T ree, our model restricted to using only Child edges without edge labels. This can be viewed as an evolution of BID17, with the difference that instead of a log-bilinear network that does not maintain state during the generation, we use a GRU. ASN is similar to abstract syntax networks BID20 and arises as an extension of the T ree model by adding edge labels on Child that encode the chosen production and the index of the child (corresponding to the "field name" Rabinovich et al. FORMULA0). Finally, Syn follows the work of , but uses a GRU instead of an LSTM. For this, we extend T ree by a new NextExp edge that connects nodes to each other in the expansion sequence of the tree, thus corresponding to the action flow .In all cases, our re-implementations improve on prior work in our variable selection mechanism, which ensures that generated programs only use variables that are defined and in scope. Both Rabinovich et al. FORMULA0 and instead use a copying mechanism from the context. On the other hand, they use RNN modules to generate function names and choose arguments from the context and to generate string literals BID20. Our ExprGen task limits the set of allowed functions and string literals substantially and thus no RNN decoder generating such things is required in our experiments. The authors of the PHOG BID5 language model kindly ran experiments on our data for the ExprGen task, to provide baseline of a non-neural language model. Note, however, that PHOG does not consider the code context to the right of the expression to generate, and does no additional analyses to determine which variable choices are valid. Extending the model to take more context into account and do some analyses to restrict choices would certainly improve its . Metrics We are interested in the ability of a model to generate valid expressions based on the current code context. To evaluate this, we consider four metrics. As our ExprGen task requires a conditional language model of code, we first consider the per-token perplexity of the model; the lower the perplexity, the better the model fits the real data distribution. We then evaluate how often the generated expression is well-typed (i.e., can be typed in the original code context). We report these metrics for the most likely expression returned by beam search decoding with beam width 5. Finally, we compute how often the ground truth expression was generated (reported for the most likely expression, as well as for the top five expressions). This measure is stricter than semantic equivalence, as an expression j > i will not match the equivalent i < j. Results We show the of our evaluation in Tab. 1. Overall, the graph encoder architecture seems to be best-suited for this task. All models learn to generate syntactically valid code (which is relatively simple in our domain). However, the different encoder models perform very differently on semantic measures such as well-typedness and the retrieval of the ground truth expression. Most of the type errors are due to usage of an "UNK" literal (for example, the G → NAG model only has 4% type error when filtering out such unknown literals). The show a clear trend that correlates better semantic with the amount of information about the partially generated programs employed by the generative models. Transferring a trained model to unseen projects with a new project-specific vocabulary substantially worsens , as expected. Overall, our NAG model, combining and adding additional signal sources, seems to perform best on most measures, and seems to be leastimpacted by the transfer. As the in the previous section suggest, the proposed ExprGen task is hard even for the strongest models we evaluated, achieving no more than 50% accuracy on the top prediction. It is also unsolvable for classical logico-deductive program synthesis systems, as the provided code context does not form a precise specification. However, we do know that most instances of the task are (easily) solvable for professional software developers, and thus believe that machine learning systems can have considerable success on the task. FIG1 shows two (abbreviated) samples from our test set, together with the predictions made by the two strongest models we evaluated. In the first example, we can see that the G → NAG model correctly identifies that the relationship between paramCount and methParamCount is important (as they appear together in the blocked guarded by the expression to generate), and thus generates comparison expressions between the two variables. The G → ASN model lacks the ability to recognize that paramCount (or any variable) was already used and thus fails to insert both relevant variables. We found this to be a common failure, often leading to suggestions using only one variable (possibly repeatedly). In the second example, both G → NAG and G → Syn have learned the common if (var. StartsWith(...)) {... var. Substring(num)... } pattern, but of course fail to produce the correct string literal in the condition. We show for all of our models for these examples, as well as for as additional examples, in the supplementary material B. We presented a generative code model that leverages known semantics of partially generated programs to direct the generative procedure. The key idea is to augment partial programs to obtain a graph, and then use graph neural networks to compute a precise representation for the partial program. This representation then helps to better guide the remainder of the generative procedure. We have shown that this approach can be used to generate small but semantically interesting expressions from very imprecise context information. The presented model could be useful in program repair scenarios (where repair proposals need to be scored, based on their context) or in the code review setting (where it could highlight very unlikely expressions). We also believe that similar models could have applications in related domains, such as semantic parsing, neural program synthesis and text generation. Below we list some sample snippets from the training set for our ExprGen task. The highlighted expressions are to be generated. On the following pages, we list some sample snippets from the test set for our ExprGen task, together with suggestions produced by different models. The highlighted expressions are the ground truth expression that should be generated.

represent the short summary for finding the corresponding scientific article (long)

Representing programs as graphs including semantics helps when generating programs

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.


Question: The inference of models, prediction of future symbols, and entropy rate estimation of discrete-time, discrete-event processes is well-worn ground. However, many time series are better conceptualized as continuous-time, discrete-event processes. Here, we provide new methods for inferring models, predicting future symbols, and estimating the entropy rate of continuous-time, discrete-event processes. The methods rely on an extension of Bayesian structural inference that takes advantage of neural network’s universal approximation power. Based on experiments with simple synthetic data, these new methods seem to be competitive with state-of- the-art methods for prediction and entropy rate estimation as long as the correct model is inferred. Much scientific data is dynamic, meaning that we see not a static image of a system but its time evolution. The additional richness of dynamic data should allow us to better understand the system, but we may not know how to process the richer data in a way that will yield new insight into the system in question. For example, we have records of when earthquakes have occurred, but still lack the ability to predict earthquakes well or estimate their intrinsic randomness ; we know which neurons have spiked when, but lack an understanding of the neural code ; and finally, we can observe organisms, but have difficulty modeling their behavior . Such examples are not only continuous-time, but also discreteevent, meaning that the observations belong to a finite set (e.g, neuron spikes or is silent) and are not better-described as a collection of real numbers. These disparate scientific problems are begging for a unified framework for inferring expressive continuous-time, discrete-event models and for using those models to make predictions and, potentially, estimate the intrinsic randomness of the system. In this paper, we present a step towards such a unified framework that takes advantage of: the inference and the predictive advantages of unifilarity-meaning that the hidden Markov model's underlying state (the so-called "causal state" or "predictive state representation" ) can be uniquely identified from the past data; and the universal approximation power of neural networks . Indeed, one could view the proposed algorithm for model inference as the continuous-time extension of Bayesian structural inference. We focus on time series that are discrete-event and inherently stochastic. In particular, we infer the most likely unifilar hidden semi-Markov model (uhsMm) given data using the Bayesian information criterion. This class of models is slightly more powerful than semi-Markov models, in which the future symbol depends only on the prior symbol, but for which the dwell time of the next symbol is drawn from a non-exponential distribution. With unifilar hidden semi-Markov models, the probability of a future symbol depends on arbitrarily long pasts of prior symbols, and the dwell time distribution for that symbol is non-exponential. Beyond just model inference, we can use the inferred model and the closed-form expressions in Ref. to estimate the process' entropy rate, and we can use the inferred states of the uhsMm to predict future input via a k-nearest neighbors approach. We compare the latter two algorithms to reasonable extensions of state-of-the-art algorithms. Our new algorithms appear competitive as long as model inference is in-class, meaning that the true model producing the data is equivalent to one of the models in our search. In Sec. 3, we introduce the reader to unifilar hidden semi-Markov models. In Sec. 4, we describe our new algorithms for model inference, entropy rate estimation, and time series prediction and test our algorithms on synthetic data that is memoryful. And in Sec. 5, we discuss potential extensions and applications of this research. There exist many methods for studying discrete-time processes. A classical technique is the autoregressive process, AR-k, in which the predicted symbol is a linear combination of previous symbols; a slight modification on this is the generalized linear model (GLM), in which the probability of a symbol is proportional to the exponential of a linear combination of previous symbols . Previous workers have also used the Baum-Welch algorithm , Bayesian structural inference , or a nonparametric extension of Bayesian structural inference to infer a hidden Markov model or probability distribution over hidden Markov models of the observed process; if the most likely state of the hidden Markov model is correctly inferred, one can use the model's structure to predict the future symbol. More recently, recurrent neural networks and reservoir computers can be trained to recreate the output of any dynamical system through simple linear or logistic regression for reservoir computers or backpropagation through time for recurrent neural networks . When it comes to continuous-time, discrete-event predictors, far less has been done. Most continuous-time data is, in fact, discrete-time data with a high time resolution; as such, one can essentially sample continuous-time, discrete-event data at high resolution and use any of the previously mentioned methods for predicting discrete-time data. Alternatively, one can represent continuoustime, discrete-event data as a list of dwell times and symbols and feed that data into either a recurrent neural network or feedforward neural network. We take a new approach: we infer continuous-time hidden Markov models and predict using the model's internal state as useful predictive features. We are given a sequence of symbols and durations of those symbols,..., (x i, τ i),..., (x 0, τ + 0). This constitutes the data, D. For example, seismic time series are of this kind: magnitude and time between earthquakes. The last seen symbol x 0 has been seen for a duration τ + 0. Had we observed the system for a longer amount of time, τ + 0 may increase. The possible symbols {x i} i are assumed to belong to a finite set A, while the interevent intervals {τ i} i are assumed to belong to (0, ∞). We assume stationarity-that the statistics of {(x i, τ i)} i are unchanging in time. Above is the description of the observed time series. What now follows is a shortened description of unifilar hidden semi-Markov models, notated M, that could be generating such a time series . The minimal such model that is consistent with the observations is the -Machine. Underlying a unifilar hidden semi-Markov model is a finite-state machine with states g, each equipped with a dwell-time distribution φ g (τ), an emission probability p(x|g), and a function + (g, x) that specifies the next hidden state when given the current hidden state g and the current emission symbol x. This model generates a time series as follows: a hidden state g is randomly chosen; a dwell time τ is chosen according to the dwell-time distribution φ g (τ); an emission symbol is chosen according to the conditional probability p(x|g); and we then observe the chosen x for τ amount of time. A new hidden state is determined via + (g, x), and we further restrict possible next emissions to be different than the previous emission-a property that makes this model unifilar-and the process repeats. See Fig. 1 for illustrations of a unifilar hidden semi-Markov model. We investigate three tasks: model inference; calculation of the differential entropy rate; and development of a predictor of future symbols. Our main claim is that restricted attention to a special type of discrete-event, continuous-time model called unifilar hidden semi-Markov models makes all three tasks easier. At left bottom, a generative model for a discrete-alphabet, continuous-time stochastic process. Dwell times τ are drawn upon transitions between states, and the corresponding symbol is shown for that amount of time. At top, the corresponding "conveyer belt" representation of the process generated by the model beneath. Conveyer belts represent the time since last symbol based on the height along the conveyer belt traveled; each conveyer belt has a symbol. To the right of the two presentations of a uhsMm, an example time series generated from the model at left, where φ A, φ B, φ C are inverse Gaussian distributions with (µ, λ) pairs of,,, respectively. The unifilar hidden semi-Markov models described earlier can be parameterized. Let M refer to a model-in this case, the underlying topology of the finite-state machine and neural networks defining the density of dwell times; let θ refer to the model's parameters, i.e. the emission probabilities and the parameters of the neural networks; and let D refer to the data, i.e., the list of emitted symbols and dwell times. Ideally, to choose a model, we would do maximum a posteriori by calculating arg max M P r(M|D) and choose parameters of that model via maximum likelihood, arg max θ P r(D|θ, M). In the case of discrete-time unifilar hidden Markov models, Strelioff and Crutchfield described the Bayesian framework for inferring the best-fit model and parameters. More than that, Ref. calculated the posterior analytically, using the unifilarity property to ease the mathematical burden. Analytic calculations in continuous-time may be possible, but we leave that for a future endeavor. We instead turn to a variety of approximations, still aided by the unifilarity of the inferred models. The main such approximation is our use of the Bayesian inference criterion (BIC). Maximum a posteriori is performed via where k M is the number of parameters θ. To choose a model, then, we must calculate not only the parameters θ that maximize the log likelihood, but the log likelihood itself. We make one further approximation for tractability involving the start state s 0, for which As the logarithm of a sum has no easy expression, we approximate Our strategy, then, is to choose the parameters θ that maximize max s0 log P r(D|s 0, θ, M) and to choose the model M that maximizes max θ log P r(D|θ, M) − k M 2 log |D|. This constitutes an inference of a model that can explain the observed data. What remains to be done, therefore, is approximation of max s0 max θ log P r(D|s 0, θ, M). The parameters θ of any given model include p(s, x|s), the probability of emitting x when in state s and transitioning to state s, and φ s (t), the interevent interval distribution of state s. Using the unifilarity of the underlying model, the sequence of x's when combined with the start state s 0 translate into a single possible sequence of hidden states s i. As such, one can show that where τ (s) j is any interevent interval produced when in state s. It is relatively easy to analytically maximize with respect to p(s, x|s), including the constraint that s,x p(s, x|s) = 1 for any s, and find that p * (s, x|s) = n(s, x|s) n(s). Now we turn to approximation of the dwell-time distributions, φ s (t). The dwell-time distribution can, in theory, be any normalized nonnegative function; inference may seem impossible. However, artificial neural networks can, with enough nodes, represent any continuous function. We therefore represent φ s (t) by a relatively shallow (here, three-layer) artificial neural network (ANN) in which nonnegativity and normalization are enforced as follows: The estimated density function from varying numbers of samples. Shown, at left, is the inferred density function using the neural network approach described here compared to the true density function (dotted, green) when given 500 samples (blue) and 5000 samples (orange). As the amount of data increases, the inferred density function becomes closer to ground truth. An interevent interval distribution with two modes was arbitrarily chosen by setting φ(τ) to a mixture of two inverse Gaussians. Shown, at right, is the mean-squared error between the estimated density and the true density as we use more training data for three different estimation techniques. The blue line denotes the ANN algorithm introduced here, in which we learn densities from a neural network; the orange line denotes the k-nearest neighbors algorithm ; and the green line uses Parzen window estimates . Our new method is competitive with the two standard methods for density estimation. • the second-to-last layer's activation functions are ReLus (max(0, x), and so with nonnegative output) and the weights to the last layer are constrained to be nonnegative; • and the output is the last layer's output divided by a numerical integration of the last layer's output. The log likelihood j log φ s (τ (s) j ) determines the cost function for the neural network. Then, the neural network can be trained using typical stochastic optimization methods. (Here, we use .) The output of the neural network can successfully estimate the interevent interval density function, given enough samples, within the interval for which there is data. See Fig. 2. Outside this interval, however, the estimated density function is not guaranteed to vanish as t → ∞, and can even grow. Stated differently, the neural networks considered here are good interpolators, but can be bad extrapolators. As such, the density function estimated by the network is taken to be 0 outside the interval for which there is data. To the best of our knowledge, this is a new approach to density estimation, referred to as ANN here. A previous approach to density estimation using neural networks learned the cumulative distribution function . Typical approaches to density estimation include k-nearest neighbor estimation techniques and Parzen window estimates, both of which need careful tuning of hyperparameters (k or h) . They are referred to here as kNN and Parzen. We compare the ANN, kNN, and Parzen approaches in inferring an interevent interval density function that we have chosen, arbitrarily, to be the mixture of inverse Gaussians shown in Fig. 2(left). The k in k-nearest neighbor estimation is chosen according to the criterion in Ref. , and h is chosen to as to maximize the pseudolikelihood. Note that as shown in Fig. 2(right), this is not a superior approach to density estimation in terms of minimization of mean-squared error, but it is parametric, so that BIC can be used. To test our new method for density estimation presented here-that is, by training a properly normalized ANN-we generated a trajectory from the unifilar hidden semi-Markov model shown in Fig. 3(left) and used BIC to infer the correct model. As BIC is a log likelihood minus a penalty for a larger number of parameters, a larger BIC suggests a higher posterior. With very little data, a two-state model shown in Fig. 3 is deemed most likely; but as the amount of data increases, the correct four-state model eventually takes precedence. See Fig. 3(right). The six-state model was never deemed more likely than a two-state or four-state model. Note that although this methodol- ogy might be extended to nonunifilar hidden semi-Markov models, the unifilarity allowed for easily computable and unique identification of dwell times to states in Eq. 5. One benefit of unifilar hidden semi-Markov models is that one can use them to obtain explicit formulae for the differential entropy rate . Such entropy rates are a measure of the inherent randomness of a process , and many have tried to find better algorithms for calculating entropy rates of complex processes (; ; ;). Setting aside the issue for now of why one would want to estimate the entropy rate, we simply ask how well one can estimate the entropy rate from finite data. Differential entropy rates are difficult to calculate directly from data, since the usual program involves calculating the entropy of trajectories of some length T and dividing by T: A better estimator, though, is the following : or the slope of the graph of vs. T. As the entropy of a mixed random variable of unknown dimension, this entropy is seemingly difficult to estimate from data. To calculate, we use a trick like that of Ref. and condition on the number of events N: We then further break the entropy into its discrete and continuous components: and use the k-nearest-neighbor entropy estimator to estimate H[τ 0:n |x 0:n, N = n], with k chosen to be 3. We estimate both H[x 0:n |N = n] and H[N] using plug-in entropy estimators, as the state space is relatively well-sampled. We call this estimator model-free, in that we need not infer a model to calculate the estimate. We introduce a model-based estimator, for which we infer a model and then use the inferred model's differential entropy rate as the differential entropy rate estimate. To calculate the differential entropy rate from the inferred model, we use a plug-in estimator based on the formula in Ref. : where the sum is over internal states of the model. The parameter µ s is merely the mean interevent interval out of state s, ∞ 0 t φ s (t)dt. We find the distribution over internal states s,p(s), by solving the linear equations We use the MAP estimate of the model as described previously and estimate the interevent interval density functions φ s (t) using a Parzen window estimate, with smoothing parameter h chosen so as to maximize the pseudolikelihood , given that those proved to have lower mean-squared error than the neural network density estimation technique in the previous subsection. In other words, we use neural network density estimation technique to choose the model, but once the model is chosen, we use the Parzen window estimates to estimate the density for purposes of estimating entropy rate. A full mathematical analysis of the bias and variance is beyond the scope of this paper. Fig. 4 shows a comparison between the model-free method (k-nearest neighbor estimator of entropy) and the model-based method (estimation using the inferred model and Eq. 11) as a function of the length of trajectories simulated for the model. In Fig. 4 (Top), the most likely (two-state) model is used for the model-based plug-in estimator of Eq. 11, as ascertained by the procedure in the previous subsection; but in Fig. 4 (Bottom), the correct four-state model is used for the plug-in estimator. Hence, the estimate in Eq. 11 is based on the wrong model, and hence, leads to a systematic overestimate of the entropy rate. When the correct four-state model is used for the plug-in estimator in Fig. 4 (Bottom), the model-based estimator has much lower variance than the model-free method. To efficiently estimate the excess entropy (; b; a), an additional important informational measure, requires models of the time-reversed process. Future research will elucidate the needed retrodictive representations of unifilar hidden semi-Markov models, which can be determined from the?forward" unifilar hidden semi-Markov models. There are a wide array of techniques developed for discrete-time prediction, as described earlier in the manuscript. We can develop continuous-time techniques that build on these discrete-time techniques, e.g. by using dwell times and symbols as inputs to a RNN. However, based on the experiments shown here, we seem to gain by first identifying continuous-time causal states. The first prediction method we call "predictive ANN" or PANN (with risk of confusion with the ANN method for density estimation described earlier) takes, as input, (x −n+1, τ −n+1),..., (x 0, τ + 0) into a feedforward neural network that is relatively shallow (six layers) and somewhat thin (25 nodes). Other network architectures were tried with little improvement. The weights of the network are trained to predict the emitted value x at time T later based on a mean-squared error loss function. For this method to work, the neural network must guess the hidden state g from the observed data, which can be done if the dwell-time distributions of the various states are dissimilar. Increases in n can increase the ability of the neural network to correctly guess its hidden state and thus predict future symbols, assuming enough data to avoid overfitting; here, n is chosen via cross-validation. The second of these methods that we will label as "RNN" will take (x −n+1, τ −n+1),..., (x 0, τ + 0) as input to an LSTM, though any RNN could have been chosen. The LSTM is asked to produce an estimate of x at time T subject to a mean-squared error loss function, similar to the first prediction method. The third of these methods that we will label as "uhsMm" preprocesses the input data using an inferred unifilar hidden semi-Markov model so that each time step is associated with a hidden state g, a time since last symbol change τ + 0, and a current emitted symbol x 0. In discrete-time applications, there is an explicit formula for the optimal predictor in terms of the -M; but for continuous-time applications, there is no such formula, and so we use a k-nearest neighbor estimate. More precisely, In orange, the model-free estimator (combination of plug-in entropy estimator and kNN entropy estimators) described in the text. In blue, the model-based estimator assuming a twostate model, i.e., the top left of Fig. 3. In black, the model-based estimator assuming a four-state model, i.e., the bottom left of Fig. 3. Lines denote the mean, and various data point denote the estimated entropy rate for different data sets of a particular size N. The model-free method has much higher variance than the model-based methods, and the model-based method for which the correct (four-state) model is used also has lower bias. Figure 5: At left, mean-squared error of the predictor of the symbol at time T from data prior to a time of 0 for when 500 data points are available and 300 epochs are used to train the ANN; at right, the mean-squared error of the predictor for when 5000 data points are available and 3000 epochs are used to train the ANN. The generating uhsMm is in Fig. 3(left). The uhsMm method infers the internal state of the unfilar hidden semi-Markov model; the PANN method uses the last n data points (x i, τ i) as input into a feedforward neural network; and the RNN method uses the past (x i, τ i) as input to an LSTM. we find the k closest data points in the training data to the data point under consideration, and estimate x T as the average of the future data points in the training set. In the limit of infinite data so that the correct model is identified, for correctly-chosen k, this method will output an optimal predictor; we choose k via cross-validation. The synthetic dataset is generated from Fig. 3 (top) with φ A (t) = φ D (t) as inverse Gaussians with mean 1 and scale 5 and with φ B (t) = φ C (t) as inverse Gaussians with mean 3 and scale 2. We chose these means and scales so that it would be easier, in principle, for the non-uhsMm methods (i.e., PANN and RNN) to implicitly infer the hidden state (A, B, C, and D). Given the difference in dwell time distributions for each of the hidden states, such implicit inference is necessary for accurate predictions. From experiments, shown in Fig. 5, the feedforward neural network and the recurrent neural network are typically outperformed by the uhsMm method. The corresponding mean-squared errors for the three methods are shown in Fig. 3(bottom) for two different dataset sizes. Different network architectures, learning rates, and number of epochs were tried; the shown in Fig. 5 are typical. Using a k-nearest neighbor estimate on the causal states (i.e., the internal state of the uhsMm) to predict the future symbol requires little hyperparameter tuning and outperforms compute-intensive feedforward and recurrent neural network approaches. We have introduced a new algorithm for inferring causal states of a continuous-time, discrete-event process using the groundwork of Ref. . We have introduced a new estimator of entropy rate that uses the causal states. And finally, we have shown that a predictor based on causal states is more accurate and less compute-heavy than other predictors. The new inference, estimation, and prediction algorithms could be used to infer a predictive model of complex continuous-time, discrete-event processes, such as animal behavior, and calculate estimates of the intrinsic randomness of such complex processes. Future research could delve into improving estimators of other time series information measures , using something more accurate than BIC to calculate MAP models, or enumerating the topology of all possible uhsMm models for non-binary alphabets (Johnson et al.).

Represent the TLDR for finding the corresponding scientific article

A new method for inferring a model of, estimating the entropy rate of, and predicting continuous-time, discrete-event processes.