Question: Represent the text to find a very short summary (1-2 sentences max).

Question: Current work on neural code synthesis consists of increasingly sophisticated architectures being trained on highly simplified domain-specific languages, using uniform sampling across program space of those languages for training. By comparison, program space for a C-like language is vast, and extremely sparsely populated in terms of `useful' functionalities; this requires a far more intelligent approach to corpus generation for effective training. We use a genetic programming approach using an iteratively retrained discriminator to produce a population suitable as labelled training data for a neural code synthesis architecture. We demonstrate that use of a discriminator-based training corpus generator, trained using only unlabelled problem specifications in classic Programming-by-Example format, greatly improves network performance compared to current uniform sampling techniques. Automated code synthesis is increasingly being studied as a way to lower the entry bar for nonexperts to create computer software, and to aid in generally taming the complexity of large-scale systems by allowing engineers to specify their intentions at a higher level of abstraction. The approach of neural code synthesis in particular has recently gained a lot of attention, applying advances in neural networks to the problem of automated synthesis. We specifically study the approach of programming by example, in which a small set of input-output examples are presented to the system to serve as a guide to the desired functionality of a program. Based on an analysis of these examples the synthesis system returns a source-code program able to replicate that functionality. Recent research in this field demonstrates promising , including and. However, research to date is limited to using domain-specific languages and often linear sequential programs without conditions or loops. We also take a neural-network-based approach to this problem in an attempt to gain inter-program inference across the training examples given to our system, potentially allowing the system to learn general aspects of programming to help synthesize new programs from unseen input/output examples. Unlike existing recent work, however, we target a general-purpose low-level programming language for code synthesis with a much larger search space of possible programs. This presents a major challenge in generating a training corpus for the neural network. Where related research has used uniform sampling methods through program search space , or even enumerative approaches , such approaches are wholly inadequate over larger search volumes -with sparse sampling producing very poor inference . To solve this training corpus generation problem we propose a novel discriminator-based system, in which new sub-corpora are iteratively created, continually measuring their functional properties against those of the problems it is attempting to solve. This process works by learning how similar the I/O mappings of generated programs are to I/O problems requested by users; by selecting programs which in increasingly similar I/O mappings we simultaneously choose programs with similar underlying source code features, until we are able to solve I/O problems requested by users. We demonstrate that the ant training corpus is greatly superior to a conventionally generated corpus via uniform sampling, when using a more generalised programming language for synthesis. We measure the performance of our approach by comparing against similar research on neural code synthesis which uses uniform or enumerative sampling for training, demonstrating that our discriminator-informed corpus generation approach far exceeds uniform sampling, by a factor of 2, in terms of find-rates. We also compare against a general baseline using genetic programming (GP); this baseline produces a surprising that GP has a broader range of programs found, although its probability of resolving any given user-provided problem is worse. Our approach offers an effective way to generate a training corpus for a high-dimensional program search space, capable of finding a wide range of unseen useful programs based only on input/output examples, without any labelled training data. At a high level our research also demonstrates that the structure of the training corpus provided to a neural network greatly affects its performance on general purpose code generation tasks, and we argue that it should therefore represent a core focus of the code synthesis community's efforts alongside work on neural network and language structures. In the remainder of this paper we firstly assess the literature in the field, focusing on neural code synthesis and specifically its corpus generation techniques. In Sec. 3 we then present the methodology we use to build our system, based on both a synthesis network and a discriminator network for corpus generation. We then evaluate our approach in Sec. 4 by comparing it against traditional training corpus generation approaches for neural code synthesis. [code to reproduce our will be made open-source should this paper be accepted, and this line will be changed to the link to the repository] Program synthesis has been studied for nearly as long as machine learning has been considered, having been proposed in 1950 by. Three main subfields exist: logic-based solvers Feng et al. (2017; 2018); Feser et al. (2015a; b);; stochastic-search-based Genetic Programming (GP);; and neural network based approaches. While solvers are excellent for restricted domains, they have limited applicability in programs containing loops due to their reliance on predicate logic. Genetic programming has been used to good effect in specific domains, but lacks the cross-program inference power that neural networks offer. We hold neural code synthesis to be an interesting area of study following the of DeepCoder , which suggests that neural network can identify higher level features of program outputs (sortedness, evenness, offset from zero mean...); combined with this is the opportunity of inter-program inference within the representational model of the neural network. Within the subfield of neural synthesis, we focus on programming by example , which takes a set of inputs and outputs which demonstrate a desired functionality. One of the core works in this field is , a system able to recognise subfunctions within a larger program consisting of up to 5 of these sub-functions called sequentially. This was significantly improved in subsequent work , which used a novel partial program execution model with fed back into the synthesis neural network, to enhance search speed and increase the length of programs that can be found up to 14. Both approaches operated in the same Domain Specific Language, which focused on arrays of integers. In terms of corpus generation, DeepCoder uses a simple enumerative strategy, as its DSL creates a small enough program space to list all possible programs, while use random sampling. Two other major domains exist in the field of neural code synthesis; In both of these, we see that synthetic training sets are predominantly generated by uniform sampling (; ;). The first of these two domains is string-manipulation functions; , as these have immediate and obvious usefulness to non-expert users. While these tasks can be handled by solvers, there are good applications for neural synthesis systems, such as , or neurally-augmented solver systems. The second domain is deduction of the behavioural algorithm of an agent in an environment, with the examples being sequences of actions taken by the agent in a grid environment. This has been studied both in terms of trained neural networks recognising behavioural patterns or in reinforcement learning based approaches to program generation. Our work differs from these domains' use of random uniform sampling for training sets, as well as from the unscalable enumerative strategy of DeepCoder, in that we introduce a strategy for generating a targeted training corpus without requirement for human-provided labelled data, and we investigate how this generation strategy significantly boosts inference performance. Finally, the most similar work to our own is , which investigate the effects of inputs to programs in terms of the produced I/O examples' effect on training success. This differs from our work in that they explore which inputs to feed into sampled programs, while we are presenting a new program sampling technique. This approach may well be complementary to our own if using as part of our training corpus generation pipeline. Our overall approach to code synthesis is to feed in a set of ten input/output (I/O) examples to a neural network, and have the output layers of that neural network select the most likely operation for each line of code for a function of a given upper length limit (where the number of output layers is equivalent to the maximum number of lines of code in the function, each neuron in an output layer is equivalent to selecting one particular operator for that line, and each line of code can be set to 'no-op'). The probabilistic nature of a neural network's output allows us to gain a set of possible programs to search through up to a given search depth; this basic approach is similar in spirit to in that it is attempting to guess the probability of each'feature' of a program -except in our case these features are relatively low-level instructions. Neural networks for code synthesis are usually trained on a uniformly sampled set of programs from the total space of all possible programs; while this is viable for simple domain-specific languages, it becomes intractable for more general purpose languages. With the aim of achieving code synthesis using a much more general programming language, our methodology focuses on how training data can be effectively drawn from a much larger search space without human input. Our approach to this uses a novel hybrid solution inspired by genetic programming but using a neural network as a fitness function. We term this neural network a discriminator, as it attempts to discriminate between the algorithms the genetic programming element is currently generating, and the likely features needed by requested I/O examples for human-useful functions. We reuse the programming language used in , a simplified C-like language, which can be compiled out to C or Java source code. The language features variable assignments, conditional operators, and loop operators. These allow complex flow-control operations, allowing us to study algorithms which go beyond linear concatenation of functions often used in the literature. We enhance the problem complexity considered by by removing the restrictions on array length, instead implementing an operation which instantiates empty arrays of given length. We also implement operations to allow array length to be determined, and the ability for a function to call itself, to allow recursive functions. Our methodology also avoids using any humanprovided hints about the likely features of source code for problem solutions, instead relying only on a set of I/O examples for unsolved problems. The full set of language operators available and restrictions used is included as an appendix 6.1. Our human useful corpus is a set of I/O mappings for 40 unique functions, each of which takes one array input parameter (of any length) and one variable, and returns an array of any length. The set of problems includes reversing arrays, appending arrays with new values, and summing the values in an array (a full list is given in appendix 6.4). We assume that these I/O mappings have been requested by human users, but that none have yet been solved. We further assume that each function has been requested at least five times, with different I/O mappings for each. This gives a total corpus of 200 I/O examples as a guide to the kinds of input-to-output transformations that are considered'useful'. We stress that at no point is the system ever provided with any source code for these examples. During early experimentation we found it beneficial to apply some conventions around these I/O examples, specifically for the first three examples in the set of ten for a given problem. The first I/O example for any problem is such that the content of each input array cell is the index of that cell, starting from 0, and the input variable has a random value. The second I/O example has the same properties but the second array cell is randomised to a new value between 8 and 8, inclusive. The third I/O example for every problem is the same as the first example except that the input variable is also randomised to the same range. The remaining seven I/O examples can be anything, and are randomly generated in our corpus. Our synthesis pipeline has the challenge of starting from this corpus of unsolved problems, specified by I/O examples, and solving as many of them as possible by synthesising the correct source code which correctly maps the given input to the given output for each problem. Two neural network architectures are used in our synthesis pipeline, a synthesiser and a discriminator. The discriminator is discussed later, and is used in generating corpora on which the synthesiser is trained. The synthesiser network is that which receives I/O examples and attempts to build a source code program to match the required functionality. This neural network has an input layer which accepts both the input and output values of each pair of 10 examples for a given problem, such that each input neuron takes a single bit of each integer. Internally we use of a set of 8 layers of 256 neurons, each connected to all previous layers, with selu activation, a simplified version of the net used in. For output, we have one output layer per line of the program to synthesise. Each output layer has one neuron for each way in which a line can be written (all valid operations), including no-op. A labelled program would be represented as an array of one-hot vectors, with the non-zero value mapping to the way that particular line should be written. We use crossentropy training loss on each line. When reading a program out, the activities of all output layers' neurons are taken and ranked, giving a confidence for each option for each line. We then search over the top 1,000,000 programs the network returns as'most confident', using a beam search technique (detailed in appendix 6.2). Our synthesiser neural network requires a training corpus comprised of the source code of example programs together with the I/O pairs for each program. Based on this training it is then able to solve some of the problems in our set of human useful I/O examples. To generate this training corpus we use an iterative process of genetic programming and discriminator training to create a series of increasingly relevant corpora. At the start of this process, we generate an initial corpus of 1,000 functionally unique programs sampled at random from the total space of all possible programs (where we quantify a program as unique if at least one output value is different from any other program when given the same five randomly-generated input parameters). Corpora other than this starting corpus are created by using a parent corpus from the set of accepted corpora (creation process detailed below). A child corpus is accepted if it finds an implementation for a human-useful IO mapping which was not found by an existing accepted lower-ancestry corpus, otherwise it is discarded and cannot be used as a parent. A corpus' ancestry is the number of parentchild relationships it is removed from the initial corpus. When creating a new child corpus, a parent corpus is selected by roulette selection from all currently accepted potential parents, with each potential parent's weighting being (0.1 + number of successf ul children)/(0.1 + number of children). After the first corpus, further corpora are generated based on the use of our discriminator. This is a neural network designed to classify input/output pairs generated by programs in our generated corpus as being closer to / further away from, those of I/O pairs in the human-useful set from users. A new corpus is created by selecting programs from a parent corpus which are measured to be most similar to human-useful programs in their input/output mappings (specifically the form of their outputs, and how these outputs seem to relate to corresponding inputs). By doing this, we hypothesise that the kinds of source code features found in these programs will similarly move closer to those needed to synthesise programs solving the human-useful I/O examples. Our discriminator neural network then works as follows. Architecturally, it consists of 2 dense layers of 16 nodes, with a single output. The featurisation of programs is identical to our synthesis neural network (as described above). We train this network by providing all of our I/O examples for all unfound human-useful target programs, and generate I/O examples for each of the 1,000 programs in the parent corpus. We train the discriminator network as a classifier to determine which of the I/O examples are from our human-useful set, and which are from the generated set. This training continues until a threshold T k is reached. T k is the proportion of programs which would be retained by the discriminator (as described below), if run on the parent corpus. T k is a randomly set value between 0.1 and 1, set as max(0.1, r 2) where r is a random real value uniformly distributed between 0 and 1. We use a random value here to increase the diversity in corpora formed, some being highly similar to the parent and some being fairly different, in a bid to maximise coverage. This trained discriminator therefore returns an estimate F d for how human-like a program is, ranging between 0 and 1. A program is said to pass the discriminator if it has an estimate of F d > 0.1. This second threshold was chosen based on preliminary experiments, particularly based on analysis of the distribution of estimates, which was found to be highly biased towards either end of the spectrum. These selected programs form the basis of a new child corpus. This child corpus is then expanded to have 1,000 functionally unique programs of its own, by using roulette wheel selection in which we take one of the existing programs in the corpus and mutate it, then accept or reject that mutated program as an additional member of the corpus based on a fitness function F q. This value F q is simply how much it exceeded the discrimination threshold (F d − 0.1). During development, roulette selection was found to produce far superior than tournament selection if the discriminator values are offset by 0.1, due to bias away from programs which only just passed the threshold. We continue to create new child corpora in this way up to a desired total number. We emphasise that, during this process, we have no access to the human-useful programs, only the I/O examples that users have requested be generated. After multiple rounds of generating self-training data in the above fashion, to reach source code features that are increasingly likely to be involved in solving the requested I/O examples, we then begin to be able to successfully synthesise solutions to the I/O examples of programs requested by users. Our synthesis pipeline returns a collection of solutions to I/O problems in the human useful set, and also returns a set of generated training corpora which were used in finding these solutions. We keep all generated corpora, and their corresponding trained synthesiser networks, which we'kept' in the above iterative process as either a parent or final child. Note that we keep the intermediate parents along the way because a child corpus is sometimes unable to solve some of the problems of its parent, even though it can solve new problems that its parent could not. These trained neural networks can then be re-used when given new I/O problems not present in the initial human useful set; each individual trained synthesiser network processes an I/O problem in ∼ 0.25 seconds. We can also collate all corpora into a single combined corpus on which to train a combined synthesiser neural network. We report the of both alternatives (the set of individual experts and the combined expert) in the following section. Our evaluation compares our approach to training corpus generation against competitive baselines. For all experiments we generate a total of 200 sub-corpora using our approach, with each experiment repeated 20 times. The HU corpus was fixed for all experiments, and the I/O examples did not vary, including in baselines, to allow consistent testing and repeatability. Noise between repeated experiments therefore derives only from the different systems' internal stochasticity. To evaluate our system we compare it against two baselines: one without using our discriminator, instead using randomly-generated corpora, and one using genetic programming on the same problem set. We then compare our approach to one using uniformly-sampled training data, as is used in related research for simpler domain-specific languages, and lastly we explore the the effects of decisions made by our discriminator in more detail. In this experiment we compare the find rates of programs against two baselines. For the first baseline we use an identical system to our own but with the discriminator removed as a fitness function, such that successive training corpora are generated only using randomly selected parents and mutations. The remainder of our pipeline is kept the same for this baseline. For our second baseline, we use genetic programming due to the inability to re-use baselines from other work in the literature, as for example seen in the DeepCoder framework, caused by the relative generality of our programming language for synthesis. Our genetic programming technique was designed to require roughly the same total computational time as the full sub-corpus generation pass, to fairly compare the options for user IO mapping resolution. It uses a population size of 1,000, and a maximum generation count of 10,000. It uses tournament selection, with a tournament size of 6 and a probability of mutation of 0.15. The fitness function is a function of the Euclidean distance between the desired output and the target output, unless the outputs differ in length, in which case it is set to an extremely negative penalty value. Our are shown in Table 1, demonstrating that the set of networks produced by discriminated sub-corpus generation between them produced the highest resolution rate for the 200 human-useful I/O mappings, returning an average of 81.5(σ = 8.3). Interestingly, the genetic programming technique found more unique functionalities. Of the 40 unique ground truth programs, the GP baseline found 22.7, while the sub-corpora found 20.3. This means that while the sub-corpus technique had a higher probability of finding a program to match a user-supplied IO mapping, there existed a stronger inequality between the find rates than the GP algorithm, which was more consistent in its probability of finding any given program. It is highly likely that this is a phenomenon produced by the nature of the discriminator itself, driving the system towards producing certain types of algorithms predominantly. It seems likely that certain properties of certain programs would be more recognisable by a neural network, and so emphasised by the discriminator's fitness function; this is a key avenue of future work. In this set of experiments we evaluate the performance of our iteratively-produced training corpus against a baseline randomly-generated training corpus. To do this we take a subset of 5 of the subcorpora produced, attempting to maximise the number of separate I/O examples found by the set of networks. The collated corpus then discards any duplicated functionalities, giving a training corpus of approximately 4,200 examples. For each of these, we write out a set of training examples, with 5 randomly generated inputs into each function being used to generate the feature I/O examples. We then train the same synthesis neural network on these as before. For the comparative baseline training data we simply sample 5,000 programs uniformly at random from the space of all possible programs, and again generate 5 training I/O examples using each of these programs. In both cases the corpora are then divided between training and validation in a 0.9:0.1 split, with validation programs being functionally distinct from those in the training set; the training data is then fed into our synthesis neural network and tested. As can be seen in Table 2, the collated corpus produced by the discriminated sub-corpus generation process more than doubles the performance of the randomly generated training corpus. The discrim- inator has driven program generation towards a set of programs which is far more representative of the types of behaviour present within the human-useful programs. Inspection of the programs within the corpora matches this expectation: there is no drive for a randomly generated program to, say, contain a loop, and if they do there is no drive towards writing to all/most cells in the output array. While the behaviour of each training program is thus distinct, this is not necessarily in a'useful' way, relative to the types of problems the user wishes to solve. The discriminator, however, can drive towards useful training programs, by emphasising programs which write relatively uniformly to output arrays, using loops and with values remaining within sensible ranges; as well as generating programs with outputs dependent on the inputs, rather than fixed-output programs. Without ever receiving training labels or information about the nature of useful features, the discriminator learns to maximise their presence in the sampled programs. In this section we explore in more detail of effect of our discriminator design. We first demonstrate the difference-over-time from our first baseline in Sec. 4.1, in which we generate successive corpora using either our discriminator or using simple random selection of parent. The of this are shown in Figure 1, in which we see how many programs from our target human useful set are solved over time as successive corpora are generated. As expected, both the discriminator setup and the random setup start at the same point, as the initial corpus does not use a discriminator. The first discriminated sub-corpus trains a network which finds an additional 6.7 solutions to I/O examples on average, compared to the non-discriminated corpus' increase of only 2.8. This progress continues as new sub-corpora are added, and is also seen on the graph of unique functionalities found. This clearly demonstrates the value of the discriminator in corpus generation. We next examine the find rates over the'ancestry' of sub-corpora in detail, to give indications as to the behaviour of the system in response to the discriminator's iterated use. Each sub-corpus past the first uses another as a parent; the ancestry of a sub-corpus is therefore the number of parents since the starting corpus. This varied by experiment, with all accepting a corpus with ancestry of at least 5, and the maximum being a single run which had a sub-corpus of ancestry 12. In Figure 2 we plot the find rates of each program over ancestry progression. We discover that certain programs are trivial to find, and do not require discriminator use (Array Length; Array to Zero...). These have a find rate of nearly 1.0 before the discriminator is used (black in first cell). Program find rates over sub-corpus ancestry. From white at 0 find rate, to fully black indicating 1.0 find rate. Border used to indicate a non-zero value. Plot cut off at ancestry = 8 due to low sample size past this point. The way actual program outputs evolve is detailed in appendix 6.3. Certain programs are found with high reliability past ancestry of 1, for example the identity program. This program was almost never found without use of the discriminator, but a single use lead to it having a nearly 1.0 find rate. This indicates that the discriminator lead to a set of subcorpora which represented the identity function's programmatic behaviour far better. We found that these sub-corpora's programs nearly universally featured loops and sequential array write operations, functionalities required to produce the identity function. The second use of the discriminator showed similar programs, such as Identity Parity and'Iterate from Start'. These were rarely found in both a non-discriminated sub-corpus, or an ancestry = 1 sub-corpus, but featured regularly at later depths. This reflects the discriminator iterating on its previous selections, attempting to discriminate between programs produced by a first-generation discriminator and the human useful corpus. These programs now found feature more complex loopusing behaviours than simple reproduction of the input array, such as using conditionals and literals. Past ancestry = 2, however, we no longer see sudden find-rate jumps. This indicates that the discriminator loses its effectiveness and can no longer guide as reliably towards the functionalities of the human-useful corpus. We speculate that the discriminator is unable to force the presence of the required functionalities in the produced training corpus either because (i) the genetic algorithm doesn't produce any for it to select; (ii) it does not have the capacity to represent the behaviours (due to low layer width or depth); (iii) these functionalities are simply not identifiable from I/O mappings alone. Further study of this is the subject of our future work. This paper has presented a discriminator-based corpus generation technique, which iteratively seeks to generate training programs drawn from the same distribution as the programs it is attempting to solve. It works without the need for labelled training data, generating its own based purely on supplied features of I/O examples and the underlying properties of the language itself. We show that it is greatly superior to one which does not use a discriminator for selecting training examples. Once generation has completed, our framework can also return a collated training corpus, allowing training of a single large neural network. We show that this collated network is also significantly stronger, in terms of quality of trained network, to one trained using random sampling techniques. Based on our , we argue that the way in which training corpora are generated for neural program synthesis deserves significant further study -and may be of equal importance to the design of the neural network used for synthesis itself. In future work we will further explore the ability of discriminator-style networks to identify specific features of code likely to be involved in solving a particular problem, as well as more advanced architectures for synthesis and discriminator networks. Our programs are implemented as a constrained version of C, as in the work of. We, however, expand upon their work, increasing the limits on variable count, and introducing the concepts of runtime-defined array lengths and recursive function calls. We limit the programs to length 14, the maximum number of integer variables to 7, the maximum number of array variables to 2 (the one input and the one ultimately returned), the maximum array length under any condition to 16 and the maximum value ever to between -255 and 255 (inclusive). The operators available are described in Table 3, and represent a superset of the operators used in the work of Wild and Porter. Each line is given a "depth" to search, ranging from a single option to up to a maximum of 10 options. The combination of all programs within this space are initially added to a set, which therefore has a size equal to the product of all the depths. The beam search constructs the initial search volume by iteratively increasing the search depth of a single line to maximise the exploration value function. This function is as follows: [a 0, a 1, a 2 ...a n] represents the ranked option confidences from the neural network, now sorted such that a 0 is the highest-confidence option. These are normalised by dividing by a 0. S is the search volume size, and S i is the search volume size if line i had its depth increased by one. D i is the current depth of search of a given line minus one (for example if D i is 0, only the first option will be added to the search volume, if it is 1, the top two ranked options will be added to the combinational set of programs (thus doubling the search space size)). The exploration value for adding increasing the depth of any given line i is This process attempts to drive the neural network towards exploring lines in which multiple options are highly confident, and away from lines where a single option has been given a high confidence and all others given a low or negative confidence. Once the iterative addition has produced a set of >= 1, 000, 000, the 1,000,000 options with the highest sum confidence are selected and searched exhaustively. In Table 4 we see outputs from 15 randomly generated programs, from 3 randomly chosen subcorpora. The first is the starting corpus of the run, which had no discriminator. The second has a discriminator trained between the human-useful IO examples and its parent corpus. The third sub-corpus then has a second generation discriminator, which was trained based on a discriminated corpus and the HU IO corpus. All outputs are responses to the function being run with an input of input array = and input integer = 2. We see that the programs in the starting corpus, ancestry=0, which were randomly sampled from program space, differ greatly from the style of program we are attempting to train the network to synthesise. The majority of all returned values are 0, and the array length varies considerably. There is little evidence that the input array is being read in, or indeed any use of loops at all. The second generation corpus, ancestry=1, shows little use of the input values, but has outputs in more consistent ranges. The output lengths now appear to always be the length of the input array, and the programs clearly use loops to write to the output array. Despite this, the output patterns are highly uniform, often being the same value repeated for most of the output array. The third generation corpus, ancestry=2, shows more complex program still. Negative values, which would require arithmetic operations to produce, are present. The values vary across larger ranges, and they show elements of the input array (the last example being the input array changed by a single element). We believe this is fairly illustrative of the behaviour of the discriminator, although more in-depth analysis of the programs produced could form a good direction for future work. Functionally, all programs within this corpus receive both an array of length 2 to 16, with values ranging between -8 and 8, and an integer also ranging from -8 to 8, the same as in the rest of the paper. Some functions simply do not use the input variable, but this is accomplished solely by ignoring it, not by changing the function input parameter set. The human useful programs and their functionalities are listed in table 5 Table 5: Human useful programs, specified by user and presented to the system as a set of IO mappings only. Absolute Returns an array of length one less than the input array. Values are shifted, such that output i = input (i+1). The first input value is therefore not replicated in the output. Returns an array of length equal to the input array. Values are shifted, such that output i = input (i+1). The last value of the output array is set to zero. Returns an array of length one greater than that of the input array. Values are the values in the input array preceded by a zero. Returns an array of length equal to that of the input array. Values are such that output i = input (i−1), with the first output value being zero. Returns an array of length equal to that of the input array. Values are the the same elements as in the input array, with the exception that all zero-values are moved to the end of the sequence. Returns an array of length equal to that of the input array. Values in output -1, 0 or 1. If input i > 0 then output i = 1, if input i < 0 then output i = −1, else output i = 0 Returns a copy of the input array, sorted in ascending order. Returns an array of length equal to that of the input array. Values are such that output i = (input i) To Iterator Returns an array of length equal to that of the input array. Values are such that output i = i Add Returns an array of length equal to that of the input array. Values are such that output i = input i + inputInteger Returns an array of length one greater than that of the input array. Values are those of the input array, followed by the input integer. Returns an array of length equal to that of the input array. Values are such that if input i > inputInteger then output i = input i else output i = inputInteger Returns an array of length equal to that of the input array. Values are such that if input i < inputInteger then output i = input i else output i = inputInteger Returns an array of length equal to that of the input array. Values are such that output i = input i +(i * inputInteger) Returns an array of length equal to that of the input array. Values are such that output i = inputInteger Returns an array of length equal to that of the input array. Values are such that if input i > inputInteger then output i = 1 else output i = −1 Iterate from Start Returns an array of length equal to that of the input array. Values are such that output i = i + inputInteger Returns an array of length equal to that of the input array. Values are such that if input i < inputInteger then output i = 1 else output i = −1 Returns an array of length equal to that of the input array. Values are such that output i = i * inputInteger Returns an array of length equal to that of the input array. Values are such that output i = input i − inputInteger

A way to generate training corpora for neural code synthesis using a discriminator trained on unlabelled data

Represent the short summary for finding the corresponding scientific article (long)!

In this paper we propose a hierarchical architecture representation in which doing random or evolutionary architecture search yields highly competitive results using fewer computational resources than the prior art.


Question: Represent the article for finding a single-sentence TLDR:

Question: Neural reading comprehension models have recently achieved impressive gener- alisation , yet still perform poorly when given adversarially selected input. Most prior work has studied semantically invariant text perturbations which cause a model’s prediction to change when it should not. In this work we focus on the complementary problem: excessive prediction undersensitivity where input text is meaningfully changed, and the model’s prediction does not change when it should. We formulate a noisy adversarial attack which searches among semantic variations of comprehension questions for which a model still erroneously pro- duces the same answer as the original question – and with an even higher prob- ability. We show that – despite comprising unanswerable questions – SQuAD2.0 and NewsQA models are vulnerable to this attack and commit a substantial frac- tion of errors on adversarially generated questions. This indicates that current models—even where they can correctly predict the answer—rely on spurious sur- face patterns and are not necessarily aware of all information provided in a given comprehension question. Developing this further, we experiment with both data augmentation and adversarial training as defence strategies: both are able to sub- stantially decrease a model’s vulnerability to undersensitivity attacks on held out evaluation data. Finally, we demonstrate that adversarially robust models gener- alise better in a biased data setting with a train/evaluation distribution mismatch; they are less prone to overly rely on predictive cues only present in the training set and outperform a conventional model in the biased data setting by up to 11% F1. Neural networks can be vulnerable to adversarial input perturbations . In Natural Language Processing (NLP), which operates on discrete symbol sequences, adversarial attacks can take a variety of forms including character perturbations , semantically invariant reformulations (b; b) or-specifically in Reading Comprehension (RC)-adversarial text insertions . A model's inability to handle adversarially chosen input text puts into perspective otherwise impressive generalisation for in-distribution test sets (; ; ; inter alia) and constitutes an important caveat to drawn regarding a model's language understanding abilities. While semantically invariant text transformations can remarkably alter a model's predictions, the converse problem of model undersensitivity is equally troublesome: a model's text input can often be drastically changed in meaning while retaining the original prediction. In particular, previous works (; a;) show that even after deletion of all but a small fraction of input words, models often produce the same output. However, such reduced inputs are usually unnatural to a human reader, and it is both unclear what behaviour we should expect from natural language models evaluated on unnatural text, and how to use such unnatural inputs to improve models. In this work, we show that in RC undersensitivity can be probed with automatically generated natural language questions. In turn, we use these to both make RC models more sensitive when they should be, and more robust in the presence of biased training data. Fig. 1 shows an examples for a BERT LARGE model ) trained on SQuAD2.0 that is given a text and a comprehension question, i.e. "What was Fort Caroline renamed to after the Spanish attack?" which it correctly answers as "San Mateo" with 98% confidence. Altering this question, however, can increase model confidence for this same prediction to 99%, even though the new question is unanswerable given the same context. That is, we observe an increase in model probability, despite removing relevant question information and replacing it with irrelevant content. We formalise the process of finding such questions as an adversarial search in a discrete input space arising from perturbations of the original question. There are two types of discrete perturbations that we consider, based on part-of-speech and named entities, with the aim of obtaining grammatical and semantically consistent alternative questions that do not accidentally have the same correct answer. We find that SQuAD2.0 and NewsQA models can be attacked on a substantial proportion of samples, even with a limited computational adversarial search budget. The observed undersensitivity correlates negatively with standard performance metrics (EM/F 1), suggesting that this phenomenon -where present -is a reflection of a model's lack of question comprehension. When training models to defend against undersensitivity attacks with data augmentation and adversarial training, we observe that they can generalise their robustness to held out evaluation data without sacrificing standard performance. Furthermore, we notice they are also more robust in a learning scenario that has dataset bias with a train/evaluation distribution mismatch, increasing their performance by up to 11%F 1. In summary, our contributions are as follows: • We propose a new type of adversarial attack targeting the undersensitivity of neural RC models, and show that current models are vulnerable to it. • We compare two defence strategies, data augmentation and adversarial training, and show their effectiveness at reducing undersensitivity errors on held-out data, without sacrificing standard performance. • We demonstrate that robust models generalise better in a biased data scenario, improving their ability to answer questions with many possible answers when trained on questions with only one. Adversarial Attacks in NLP. Adversarial examples have been studied extensively in NLP -see for a recent survey. However, automatically generating adversarial examples in NLP is non-trivial, as the search space is discrete and altering a single word can easily change the semantics of an instance or render it incoherent. Recent work overcomes this issue by focusing on simple semantic-invariant transformations, showing that neural models can be oversensitive to such modifications of the inputs. For instance, Ribeiro et al. (2018b) use a set of simple perturbations such as replacing Who is with Who's. Other semantics-preserving perturbations include typos , the addition of distracting sentences , character-level adversarial perturbations , and paraphrasing (a). In this work, we instead focus on undersensitivity of neural RC models to semantic perturbations of the input. This is related to previous works leveraging domain knowledge for the generation of adversarial examples : our method is based on the idea that modifying, for instance, the named entities involved in a question can completely change its meaning and, as a consequence, the answer to the question should also differ. Our approach does not assume white-box access to the model, as do e.g. and. Undersensitivity. demonstrated classifier undersensitivity in computer vision, where altered input images can still produce the same prediction scores, achieved using (approximately) invertible networks. investigated over-and undersensitivity in dialogue models and addressed the problem with a max-margin training approach. Ribeiro et al. (2018a) describe a general model diagnosis tool to identify minimal sufficient feature sets that are sufficient for a model to form high-confidence predictions. showed that it is possible to reduce inputs to minimal input word sequences without changing a model's predictions. We see our work as a continuation of this line of inquiry, but with a particular focus on undersensitivity in RC. In contrast to , we consider concrete alternative questions, rather than arbitrarily reduced input word sequences. We furthermore address the observed undersensitivity using dedicated training objectives, in contrast to and Ribeiro et al. (2018a) who simply highlight it. Finally, one of the baseline methods we later test for defence against under-sensitivity attacks is a form of data augmentation that has similarly been used for de-biasing NLP models ). Unanswerable Questions in Reading Comprehension.'s publication of adversarial attacks on the SQuAD1.1 dataset, proposed the SQuAD2.0 dataset, which includes over 43,000 human-curated unanswerable questions. A second dataset with unanswerable question is NewsQA , comprising questions about news texts. Training on these datasets should in models with an ability to tell whether questions are answerable or not; we will see, however, that this does not extend to adversarially chosen unanswerable questions in our undersensitivity attacks. address unanswerability of questions from a given text using additional verification steps. Other approaches have shown the benefit of synthetic data to improve performance in SQuAD2.0 . We operate on the same underlying research premise that the ability to handle unanswerable questions is an important part of improving text comprehension models. In contrast to prior work, we demonstrate that despite improving performance on test sets that include unanswerable questions, the problem persists when adversarially choosing from a larger space of questions. Problem Overview. Consider a discriminative model f θ, parameterised by a collection of dense vectors θ, which transforms an input x into a predictionŷ = f θ (x). In our task, x = (t, q) is a given text t paired with a question q about this text. The label y is the answer to q where it exists, or a NoAnswer label where it cannot be answered. In a text comprehension setting with a very large set of possible answers, predictionsŷ should be specific to x, i.e. not the model prediction for arbitrary inputs. And indeed, randomly choosing a different input (t, q) is usually associated with a change of the model predictionŷ. However, there exist many examples where the prediction erroneously remains stable; the goal of the attack formulated here is to find such cases. Concretely, given a computational search budget, the goal is to discover inputs x, for which the model still erroneously predicts f θ (x) = f θ (x), even though x is not answerable from the text. Input Perturbation Spaces. Identifying suitable candidates for x can be achieved in manifold ways. A simple option is to search among a large question collection, but we find this approach to only rarely be successful; an example is shown in Table 8, Appendix C. Generating x, on the other hand, is prone to in ungrammatical or otherwise infeasible text. Instead, we consider a perturbation space X T (x) spanned by perturbing original inputs x using a perturbation function family T: This space X T (x) contains alternative model inputs derived from x. Ideally the transformation function family T is chosen such that the correct label of these new inputs is changed: for x ∈ X T (x): y(x) = y(x). We will later search in X T (x) to find inputs x which erroneously retain the same prediction as x:ŷ(x) =ŷ(x). Part-of-Speech (PoS) Perturbations. We first consider the perturbation space X T P (x) generated by PoS perturbations T P of the original question: we swap individual tokens with other, PoSconsistent alternative tokens, where we draw from large collections of tokens of the same PoS types. For example, we might alter the question Who patronized the monks in Italy? to Who betrayed the monks in Italy? by replacing the past tense verb patronized with betrayed. There is no guarantee that the altered question will require a different answer (e.g. due to synonyms). Even more so -there might be type clashes or other semantic inconsistencies (e.g. Who built the monks in Italy?). We perform a qualitative analysis to investigate the extent of this problem and find that, while a valid concern, for the majority of attackable samples there exist attacks based on correct well-formed questions (see Section 5). Named Entity Perturbations. The space X T E (x) generated by the transformation family T E is created by substituting mentions of named entities in the question with different type-consistent named entities, derived from a large collection E. For example, a comprehension question Who patronized the monks in Italy? could be altered to Who patronized the monks in Las Vegas?, replacing the geopolitical entity Italy with Las Vegas, chosen from E. Altering named entities often changes the specifics of the question and poses different requirements to the answer, which are unlikely to be satisfied from what is stated in the given text t, given the broad nature of possible entities in E. While it is not guaranteed that perturbed questions are in fact unanswerable or require a different answer, we will find in a following qualitative analysis that in the large majority of cases they do. Undersensitivity Attacks. Thus far we have described different methods of perturbing questions. We will search in the ing perturbation spaces X T P (x) and X T E (x) for inputs x for which the model prediction remains constant. However, we pose a slightly stronger requirement: f θ should assign an even higher probability to the same predictionŷ(x) =ŷ(x) than for the original input: Note that this is a conservative choice, guaranteed to preserve the prediction. To summarise, we are searching in a perturbation space for altered questions which in a higher model probability to the same answer as the original input question. If we have found such an altered question that satisfies the inequality, we have identified a successful adversarial attack, which we will refer to as an undersensitivity attack. Adversarial Search in Perturbation Space. In its simplest form, a search for an adversarial attack in the previously defined attack spaces amounts to a search over a list of single lexical alterations for the maximum (or any) higher prediction probability. We can however recur the replacement procedure multiple times, arriving at texts with potentially larger lexical distance to the original question. For example, in two iterations of PoS-consistent lexical replacement, we can alter Who was the duke in the battle of Hastings? to inputs like Who was the duke in the expedition of Roger? The space of possibilities with increasing distance grows combinatorially, and with increasing perturbation radius it becomes computationally infeasible to comprehensively cover the full perturbation spaces arising from iterated substitutions. To address this, we follow and apply beam search to narrow the search space, and seek to maximise the difference Beam search is conducted up to a pre-specified maximum perturbation radius ρ, but once x with ∆ > 0 has been found, we stop the search. Relation to Attacks in Prior Work. Note that this type of attack stands in contrast to other attacks based on small, semantically invariant input perturbations (; ; b) which investigate oversensitivity problems. Such semantic invariance comes with stronger requirements and relies on synonym dictionaries or paraphrases harvested from back-translation (b), which are both incomplete and noisy. Our attack is instead focused on undersensitivity, i.e. where the model is stable in its prediction even though it should not be. Consequently the requirements are not as difficult to fulfil when defining perturbation spaces that alter the question meaning, and one can rely on sets of entities and PoS examples automatically extracted from a large text collection. In contrast to prior attacks , we evaluate each perturbed input with a standard forward pass rather than using a first-order Taylor approximation to estimate the output change induced by a change in input. This is less efficient but exact, and furthermore does not require white-box access to the model and its parameters. Training and Dataset Details. We next conduct experiments using the attacks laid out above to investigate model undersensitivity. We attack the BERT model fine-tuned on SQuAD2.0 , and measure to what extent the model exhibits undersensitivity when adversarially choosing input perturbations. Note that SQuAD2.0 per design contains unanswerable questions in both training and evaluation sets; models are thus trained to predict a NoAnswer option where a comprehension question cannot be answered. In a preliminary pilot experiment, we first train a BERT LARGE model on the full training set for 2 epochs, where it reaches, 78.32%EM and 81.44%F 1, in close range to reported by. We then however choose a different training setup as we would like to conduct adversarial attacks on data entirely inaccessible during training: we split off 5% from the original training set for development purposes and retain the remaining 95% for training, stratified by articles. We use this development data to tune hyperparameters and perform early stopping, evaluated every 5000 steps with batch size 16 and patience 5, and will later tune hyperparameters for defence on it. The original SQuAD2.0 development set is then used as evaluation data, where the model reaches 73.0%EM and 76.5%F 1; we will compute the undersensitivity attacks on this entirely held out part of the dataset. Attack Details. To compute the perturbation spaces, we collect large sets of string expressions across Named Entity and PoS types to define the perturbation spaces T E and T P, which we gather from the Wikipedia paragraphs used in the SQuAD2.0 training set, with the pretrained taggers in spacy 2, and the Penn Treebank tag set for PoS. This on average in 5126 different entities per entity type, and 2337 different tokens per PoS tag. When computing PoS perturbations, we found it useful to disregard perturbations of particular PoS types that often led to only minor changes or incorrectly formed expressions, such as punctuation or determiners; more details on the left out tags can be found in Appendix A. As the number of possible perturbations to consider is potentially very large, we limit beam search at each step to a maximum of η randomly chosen type-consistent entities from E, or tokens from P, and re-sample these throughout the search. We use a beam width of b = 5, ing in a bound to the total computation spent on adversarial search of b · ρ · η model evaluations per sample, where ρ is the perturbation'radius' (the maximum search depth). Metric: Adversarial Error Rate. We quantify adversarial vulnerability to the described attacks by measuring the proportion of evaluation samples for which at least one undersensitivity attack is found given a computational search budget, disregarding cases where a model predicts NoAnswer. budgets (η = 32, ρ = 1) reach more than 60% attack success rates, and this number can be raised to 95% with a larger computational budget. For perturbations based on Named Entity substitution, we find overall lower attack success rates, but still find that more than half of the samples can successfully be attacked under the budgets tested. Note that where attacks were found, we observed that there often exist multiple alternatives with higher probability. These findings demonstrate that BERT is not necessarily specific about the entire contents of a comprehension question given to it, and that even though trained to tell when questions are unanswerable, the model often fails when facing adversarially selected unanswerable questions. In a side experiment we investigated undersensitivity attacks using Named Entity perturbations on SQuAD1.1, which prove even more vulnerable with an adversarial error rate of 70% already with a budget of η = 32; ρ = 1 (compared to 34% on SQuAD2.0). While this demonstrates that undersensitivity is also an issue for SQuAD1.1, the unanswerable question behaviour is not really well-defined, making hard to interpret. On the other hand, the notable drop between the datasets demonstrates the effectiveness of the unanswerable questions added during training in SQuAD2.0. Qualitative Analysis of the Attacks. As pointed out before, the attacks are noisy as the introduced substitutions are by no means guaranteed to in meaningful and semantically consistent expressions, or require a different answer than the original. To gauge the extent of this we inspect 100 successful attacks conducted at ρ = 6 and η = 256 on SQuAD2.0, both for PoS perturbations and named entity perturbations. We label them as either 1. Having a syntax error (e.g. What would platform lower if there were fewer people?). These are mostly due to cascading errors stemming from wrong named entity/PoS tag predictions. 2. Semantically incoherent (e.g. Who built the monks?) 3. Questions that require the same correct answer as the original, e.g. due to a paraphrase. 4. Valid attacks: Questions that would either demand a different answer or are unanswerable given the text (e.g. When did the United States withdraw from the Bretton Woods Accord? and its perturbed version When did Tuvalu withdraw from the Bretton Woods Accord?) Table 1 shows several example attacks along with their annotations, and in Table 2 perturbed version What scorer did the case go before the supreme court?), it is remarkable that the model assigns higher probabilities to the original answer even when faced with incoherent questions, casting doubt on the extent of question information being used to determine the answer. Since the named entity-based attacks have a substantially larger fraction of valid, well-posed alternative questions, we will focus our study on these attacks for the remainder of this paper. We found that models are vulnerable to undersensitivity adversaries, however not all samples were successfully attacked. This raises questions on what distinguishes samples that can and cannot be attacked. We investigate various characteristics, aiming to understand model vulnerability causes. Questions that can be attacked produce lower original prediction probabilities, with an average of 72.9% compared to 83.8% for unattackable questions. That is, there exists a direct inverse link between a model's original prediction probability and sample vulnerability to an undersensitivity attack. The adversarially chosen questions had an average probability of 78.2%, i.e. a notable gap to the original questions. It is worth noting that search halted once a single question with higher probability was found. Vulnerable samples are also less likely to be given the correct prediction overall. Concretely, evaluation metrics for vulnerable examples are 56.4%/69.6% EM/F 1, compared to 73.0%/76.5% on the whole dataset. Attackable questions have on average 12.3 tokens, whereas unattackable ones are slightly shorter with on average 11.1 tokens. We considered the distribution of different question types (What, Who, When, ...) for both attackable and unattackable samples and did not observe notable differences apart from the single most frequent question type What; it is a lot more prevalent among the unattacked questions (56.4%) than under successfully attacked questions (42.1%). This is by far the most common question type, and furthermore one that is comparatively open-ended and does not prescribe particular type expectations to its answer, as e.g., a where question would require a location. A possible explanation for the prevalence of the What questions among defended samples is thus, that the model cannot rely on type constraints alone to arrive at its predictions, and is thus less prone to such exploitation. Section 6.2 will address this in more detail. Fig. 3 shows a histogram of the 10 most common named entity tags appearing in unattackable samples versus the corresponding fraction of replaced entities in successfully attacked samples. Besides one exception, the distributions are remarkably similar. Undersensitivity can be induced for a variety of entity types used in the perturbation, but in particular questions with geopolitical entities (GPE) are error-prone. A possible explanation is observations on (non-contextualised) word embeddings clustering geopolitical entities (e.g. countries) close to one another, thus making them potentially hard to distinguish for a model operating on these embeddings . We will now investigate methods for mitigating excessive model undersensitivity. Prior work has considered both data augmentation and adversarial training for more robust models, and we will conduct experiments with both. Adding a robustness objective can negatively impact standard test metrics , and it should be noted that there exists a natural trade-off between performance on one particular test set and performance on a dataset of adversarial inputs. We perform data augmentation and adversarial training by adding a corresponding loss term to the standard log-likelihood training objective: where Ω is the standard training data, fit with a discriminative log-likelihood objective, Ω either a set of augmentation data points, or of successful adversarial attacks where they exist, and λ > 0 a hyperparameter. In data augmentation, we randomly sample perturbed input questions, whereas in adversarial training we perform an adversarial search to identify them (η = 32, ρ = 1). In both cases, alternative data points in Ω will be fit to a NULL label to represent the NoAnswer prediction -which is also fit with a log-likelihood objective. Note that we continuously update Ω throughout training to reflect adversarial samples w.r.t the current model. Experimental Setup: SQuAD2.0. We train the BERT LARGE model on SQuAD2.0, tuning the hyperparameter λ ∈ {0.0, 0.01, 0.1, 0.25, 0.5, 0.75, 1.0, 2.0} and find λ = 0.25 to work best for either of the two defence strategies. We tune the threshold for predicting NoAnswer based on validation data and report on the test set (original SQuAD2.0 Dev set). All experiments are run with batch size 16, named entity perturbations for the defence methods, and a relatively cheap adversarial attack with budget η = 32 and ρ = 1. Where no attack is found for a given question we redraw standard samples from the original training data. We evaluate the model on its validation data every 5000 steps (batch size 16) and perform early stopping with a patience of 5. Experimental Setup: NewsQA. Following the experimental protocol for SQuAD, we further test a BERT BASE model on NewsQA , which -like SQuAD2.0 -contains unanswerable questions. As annotators do often not fully agree on their annotation in NewsQA, we opt for a conservative choice and filter the dataset, such that only samples with the same majority annotation are retained, following the preprocessing pipeline of. Experimental Outcomes. Results for these experiments can be found in Table 3 and Table 4 for the two datasets, respectively. First, we observe that both data augmentation and adversarial training substantially reduce the number of undersensitivity errors the model commits, consistently across adversarial search budgets, and consistently across the two datasets. This demonstrates that both training methods are effective defences and can mitigate -but not eliminate -the model's undersensitivity problem. Notably the improved robustness -especially for data augmentationis possible without sacrificing performance in the overall standard metrics EM and F 1, even slight improvements are possible. Table 4: Breakdown of undersensitivity error rate overall (lower is better), and standard performance metrics (EM, F 1 ; higher is better) on different subsets of NewsQA evaluation data, all in [%]. Second, data augmentation is a more effective defence training strategy than adversarial training. This holds true both in terms of standard and adversarial metrics, and hints potentially at some adversarial overfitting on the training set. Finally, a closer inspection of how performance changes on answerable (HasAns) vs. unanswerable (NoAns) samples of the datasets reveals that models with modified training objectives show improved performance on unanswerable samples, while sacrificing some performance on answerable samples. 4 This suggests that the trained models -even though similar in standard metrics -evolve on different alleyways during training, and the modified objective prioritise fitting unanswerable questions to a higher degree. In Tables 3 and 4 are computed using the same perturbations at training and evaluation time. The perturbation space is relatively large, and questions are about a disjoint set of articles at evaluation time. Nevertheless there is the potential of overfitting to the particular perturbations used during training. To measure the extent to which the defences generalise also to new, held out sets of perturbations, we assembled a new, disjoint perturbation space of identical size per NE tag as those used during training, and evaluate models on attacks with respect to these perturbations. Named entities are chosen from English Wikipedia using the same method as for the training perturbation spaces, and chosen such that they are disjoint from the training perturbation space. We then ran adversarial attacks using these new attack spaces on the previously trained models, and found that both vulnerability rates of the standard model, as well as relative defence success transfer to the new attack spaces. For example, with η = 256 we observed vulnerability ratios of 51.7%, 20.7%, and 23.8% on SQuAD2 for standard training, data augmentation, and adversarial training, respectively. Detailed for different values of η, as well as for NewsQA can be found in Appendix B. Datasets for high-level NLP tasks often come with annotation and selection biases; models then learn to exploit shortcut triggers which are dataset-but not task-specific . For example, a model might be confronted with questions/paragraph pairs which only ever contain one type-consistent answer span, e.g. mention one number in a text with a'How many...?' question. It is then sufficient to learn to pick out numbers from text to solve the task, irrespective of other information given in the question. Such a model might then have trouble generalising to articles that mention several numbers, as it never learned that it is necessary to take into account other relevant question information that helps determine the correct answer. We test models trained in such a scenario: a model is trained on SQuAD1.1 questions with paragraphs containing only a single type-consistent answer expression for either a person, date, or numerical answer. At test time, we present it with question/article pairs of the same respective question types, but now there are multiple possible type-consistent answers in the paragraph. We obtain such data from , who first described this biased data scenario. We use the same training data, but split the test set with a 40/60% split 5 into development and test data. 6 We then test both a vanilla fine-tuned BERT BASE transformer model, and a model trained to be less vulnerable to undersensitivity attacks using data augmentation. Finally, we perform a control experiment, where we join and shuffle all data points from train/dev/test (of each question type, respectively), and split the dataset into new parts of the same size as before, which now follow the same data distribution (w/o data bias setting). Table 5 shows the . In this biased data scenario we observe a marked improvement across metrics and answer type categories when a model is trained with unanswerable samples. This demonstrates that the negative training signal stemming from related -but unanswerable -questions counterbalances the signal from answerable questions in such a way, that the model learns to better take into account the relevant information in the question, which allows it to correctly distinguish among several type-consistent answer possibilities in the text, which the standard BERT BASE model does not learn well. We next evaluated BERT LARGE and BERT LARGE + Augmentation Training on ADDSENT and ADDONESENT, which contain adversarially composed samples . Our , summarised in Table 10 in the Appendix, show that BERT LARGE with robust training improves both EM and F1 on both datasets, boosting F1 by 3.7 and 1.6 points on the two datasets, respectively. We trained a RoBERTa model on SQuAD2, and conducted undersensitivity attacks (ρ = 6, η = 256). Attack rates are lower for RoBERTa (34.5%), and when considering only samples where RoBERTa was found vulnerable, BERT also has a vulnerability rate of 90.7%. Concrete adversarial inputs chosen for Roberta transfer when evaluating BERT transfer for 17.5% of samples. We have investigated a problematic behaviour of RC models -being overly stable in their predictions when given semantically altered questions. This undersensitivity can be drastically reduced with appropriate defences, such as adversarial training, and in more robust models without sacrificing standard performance. Future work should study in more detail the causes and better defences to model undersensitivity, which we believe provides an alternative viewpoint on evaluating a model's RC capabilities. 5 approximate as we stratify by article 6 We also include an experiment with the setup used in Table 7: Breakdown of undersensitivity error rate on NewsQA with a held-out attack space (lower is better). A APPENDIX: POS PERTURBATION DETAILS. We exclude these PoS-tags when computing perturbations: Vulnerability for new, held-out perturbation spaces, disjoint from those used during training, can be found in Table 6 for SQuAD2, and in 7 for NewsQA. Searching in a large collection of (mostly unrelated) natural language questions, e.g. among all questions in the SQuAD training set, yields several cases where the prediction of the model increases, compared to the original question, see Table 8 for one example. Such cases are however rare, and we found the yield of this type of search to be very low. D APPENDIX: ATTACK EXAMPLES Table 11 shows more examples of successful adversarial attacks for NER perturbations on SQuAD2.0. E APPENDIX: BIASED DATA SETUP For completeness and direct comparability, we also include an experiment with the same data setup chosen in (not holding aside a dedicated validation set). Results can be found in Table 9. We again observe improvements in the biased data setting, and the robust model outperforms GQA in two of the three subtasks. F APPENDIX: VULNERABILITY ANALYSIS ON NEWSQA Fig. 4 depicts the vulnerability of a BERT LARGE model on NewsQA under attacks using named entity perturbations.

Represent the TLDR for finding the corresponding scientific article

We demonstrate vulnerability to undersensitivity attacks in SQuAD2.0 and NewsQA neural reading comprehension models, where the model predicts the same answer with increased confidence to adversarially chosen questions, and compare defence strategies.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We introduce and analyze the phenomenon of "hallucinations" in NMT, or spurious translations unrelated to source text, and propose methods to reduce its frequency.