Question: Represent the scientific article to find a short TLDR.

Question: We present the first end-to-end verifier of audio classifiers. Compared to existing methods, our approach enables analysis of both, the entire audio processing stage as well as recurrent neural network architectures (e.g., LSTM). The audio processing is verified using novel convex relaxations tailored to feature extraction operations used in audio (e.g., Fast Fourier Transform) while recurrent architectures are certified via a novel binary relaxation for the recurrent unit update. We show the verifier scales to large networks while computing significantly tighter bounds than existing methods for common audio classification benchmarks: on the challenging Google Speech Commands dataset we certify 95% more inputs than the interval approximation (only prior scalable method), for a perturbation of -90dB. Recent advances in deep learning have enabled replacement of traditional voice recognition systems with a single neural network trained from data (; ;). Wide adoption of these networks in consumer devices poses a threat to their safety when exposed to a malicious adversary. Indeed, it was recently shown that an adversary can inject noise unrecognizable to a human and force the network to misclassify (; ; ; ; ; ; ; ;), exposing a serious security flaw. Ideally, when deploying an automated speech recognition system we would like to guarantee that the system is robust against noise injected by an adversary. There has been substantial recent work on certifying robustness of computer vision models (; ; ; ; ; ; ; ; ; ; ; ; ; 2019a; ; b). However, the audio domain poses unique challenges not addressed by prior certification work for vision. Differences between audio and vision models Concretely, while an input to a vision model is a raw image, audio models typically come with a complex preprocessing stage (that involves non-trivial non-linear operations such as logarithm) which extracts relevant features from the signal. Additionally, audio systems typically use recurrent architectures which computer vision verifiers do not handle as they focus on fully-connected, convolutional and residual architectures. This work We address both of these challenges and propose an end-to-end verification method for neural network based audio classifiers and an implementation of this method in a system called DAC (Deep Audio Certifier). Our threat model assumes an attacker can introduce a noise-based perturbation to the raw audio input signal. The goal then is to certify that, for any signal that the attacker can produce, the neural network classifies the signal to the correct label. We perform verification of this property using the framework of abstract interpretation . At a high level, the idea is to maintain an abstraction capturing all possible behaviors of both the audio processing stage and the neural network. The flow of DAC is shown in Fig. 1 where all abstractions are dark blue shapes. Here, all possible signals an attacker can obtain are captured using an abstraction s (i) (a convex relaxation). This abstraction is then propagated through the audio processing stage (shown in green boxes). The key components of this step are abstract transformers. For each audio processing operation (e.g. FFT) we create an abstract transformer which receives an abstraction representing an approximation of all possible inputs to the operation and outputs a new abstraction which approximates all possible outputs of the operation. The of the audio processing stage is the abstraction x (i). The shape x (i) is then used as input to the recurrent LSTM unit (light blue) which maintains an abstraction of a hidden state h (i−1). LSTM consists of multiple operations and we create a custom abstract transformer for each of those. The of the transformers in LSTM is a new hidden state h (i). If this was the last frame in the signal (meaning i = T), then hidden state h (T) is passed through the fully connected layer of the neural network and, again using the abstract transformer, the final abstract shape a is obtained at the output (at the right of Fig. 1). Finally, to certify the property we check if each concrete output in the abstraction a classifies to the correct label (this is typically easy). If this is true, the output of the network is correct for all inputs that the attacker can create. Related work on RNN certification The work of proposes the POPQORN verifier for recurrent neural networks (RNN). We note that POPQORN does not handle the audio preprocessing pipeline. Even though POPQORN cannot directly verify audio classifiers, their approximations for LSTM non-linearities can be integrated in DAC. This in ≈ 200× slowdown with small decrease in the volume of the approximation. The massive slowdown makes their approximations unsuitable for certifying audio classifiers. In contrast, using our custom abstract transformers for LSTM non-linearities, DAC can precisely certify end-to-end robustness of challenging audio classifiers in few minutes. Our main contributions are: 1. A novel and efficient method to certify robustness of neural network audio classifiers to noise-based perturbations. The method is based on new abstract transformers which handle non-linear operations used in both audio processing and recurrent architectures. 2. An implementation of both verification and provably robust training in a system called DAC. We evaluated DAC on common audio classification benchmarks, showing it scales to realistic networks and is far more precise (97% to 2%) than the next best scalable method. We first define a threat model that we work with and then present all operations that are part of the verification procedure, including audio processing (MFCC) and LSTM updates. We also discuss the type of verification method we employ. Threat model We follow the same attacker threat model as. The assumption is that the attacker can add noise δ to the original signal s so to obtain a perturbed signal s = s + δ. The measure of signal distortion are decibels (dB) defined as: Note that the quieter the noise is, the smaller the decibel of perturbation dB s (δ) (it is usually a negative value as it is quieter than the signal). We assume the attacker can generate noise δ such that dB s (δ) < where is a constant defined by the threat model. Our goal is to verify whether the neural network classifies s correctly for each small perturbation δ (as constrained above). Though there have been number of works which operate directly on the raw signal , Mel-Frequency Cepstrum (MFC) is traditionally preferred for audio preprocessing in speech recognition systems e.g., DeepSpeech . The idea of MFC is to model non-linear human acoustic perception as power spectrum filters based on certain frequencies, called Mel-frequencies. The final of the transformation is a vector of coefficients whose elements contain log-scaled values of filtered spectra, one for every Mel-frequency. This ing vector is a feature representation of the original signal and can now be used in a downstream task such as audio classification. presented an approach to represent MFCC computation using several matrix operations which we integrate with our verification framework. Given T frames of audio signals of tr ∈ R T ×N (tr for transpose), audio preprocessing with MFCC is calculated using the following steps: 1. Pre-emphasizing and Windowing: Y = S(I N − c pe I +1 N) H Transform the signal with the pre-emphasizing and applying the Hamming window. Here, I +1 N ∈ R N ×N is the shifted diagonal identity matrix (I T ×N is the Hamming window, and c pe is the pre-emphasizing constant. 2. Power Spectrum of Fast Fourier Transform (FFT): Perform FFT on the windowed data and square it to get the real-value spectrum. We can denote FFT on discrete domain (DFT) with the multiplication of Y and W ∈ C N ×N/2. 3. Filter Bank Log Energy: Ψ = log(ΘΛ) Apply the Mel frequency filter bank to the power spectrum and get the log value of them. Λ ∈ R N/2×p is the filter bank given the number of filters p, and log is applied entry-wise. 4. DCT(Discrete Cosine Transformation): X = ΨD Perform DCT on the previous . Again, this can be formulated using matrix multiplication. We use the ing tr as the input for the neural network. Long-Short Term Memory (LSTM) LSTM architectures are a key part in modern state-of-the-art speech recognition systems . In our work, we consider the following definitions of updates in the LSTM unit. where [·, ·] is the horizontal concatenation two row vectors, W · and b · are kernel and bias of the cell, respectively. At timestep t, vectors f 0 represent pre-activations of the forget, input, and output gate, respectively, and the pre-calculation of cell state. Cell state c (t) and hidden state h (t) computed at timestep t are propagated to the LSTM unit at the next timestep, thus allowing it to maintain states. This recurrent architecture allows inputs with arbitrary length, making it especially suited for audio inputs. Robustness certification In this work, our goal will be to certify the robustness of an audio classification pipeline (including the LSTM) to noise perturbations of the input. To build such a verifier, we leverage the general method of abstract interpretation suggested by , successfully employed by some of the most recent state-of-the-art verifiers of neural networks (; a). The basic idea here will be to propagate the possible perturbations (captured by a convex region) through the operations of the entire audio pipeline and to then use the final output region to certify the robustness property. The most challenging step of this approach is defining efficient yet precise approximations of the non-linear operations (called abstract transformers) used in the audio pipeline. In this work, we will introduce a number of such new abstract transformers that handle the non-linear operations used in audio processing. Specifically, our over-approximations will be expressed in the recent DEEPPOLY abstraction (a) (a restricted form of convex polyhedra) which aims to balance efficiency and precision. That is, the abstract transformer of a non-linear function will take as input a convex polyhedra element expressible in DEEPPOLY and output another polyhedra which over-approximates the behavior of the non-linear operation when invoked with any concrete point inside the original polyhedra. We now explain the workings of our verifier on a (toy) example from the point where sound enters the processing stage to where the output of the classifier is certified as robust under any allowed perturbation of the threat model. Our goal is to provide an intuitive understanding, formal details are provided later. Audio preprocessing Unlike in standard computer vision tasks where the input is fed directly to the neural network, audio models typically first perform MFCC preprocessing to extract useful features 2 + 1.85 2 + 2.50 from the signal. We represent all preprocessing operations as green boxes in Fig. 2. The calculation follows steps described formally in Section 2, using the same notation for ing vectors. The first operation is Fast Fourier Transform (FFT) of the pre-emphasized input signal. It is a two-step process shown in dashed boxes in Fig. 2. We decompose it into affine and square operations which transform signal s (t) into an intermediate representation θ (t). Using our novel and optimal abstract transformer for the square operation, formally defined in Section 4 and visualized in Fig. 3b, we obtain linear bounds on θ (t). The next operation is Filterbank transform (FB) which consists of affine and logarithm operations, shown in solid boxes in Fig. 2. Note that if the approximations from the square transformer allows negative values, then the entire analysis will fail as the logarithm operation is undefined for negative inputs. Our transformers are carefully designed to avoid this scenario. To obtain bounds on the output of the logarithm we apply our novel and optimal logarithm transformer, also formally described in Section 4 and visualized in Fig. 3a, and obtain linear upper and lower bounds on the log energy of the filtered power spectrum, ψ (t). Logarithm operation is followed by Discrete Cosine Transform (DCT) ing in a vectorx (t) which is then used as an input to the fully connected layer of the neural network followed by a ReLU. Our analysis (detailed calculation in Appendix C) producesx 0.17, 12.8]. Since all the values are positive, the following ReLU has no effect on its input and thus we set x (t) =x (t). Using the back-substitution technique we describe later, we derive x 0.17, 12.83]. In the next paragraph we describe bound propagation through LSTM in more detail. LSTM bound propagation Here we provide a technical overview of our LSTM transformer, formalized in Section 5 and visualized in Fig. 4, by calculating the of the transformation on the first neuron of the LSTM hidden layer. We provide detailed mathematical basis of this process in Appendix D. For our toy example, let the gates be updated as f 2 /4. Also, assume the previous states are bounded by: ∈ [0.90, 1.00]. We now apply our σ(x) · tanh(y) and σ(x) · y transformers to get bounds of the cell state c Summing up the inequalities above in c 1 ), we again apply our abstract transformer and obtain 0.48c (a) Log abstract transformer. Figure 3: Our DeepPoly approximations of (a) the natural logarithm and (b) the square function. Blue lines represent the valid (in case of square, non-negative) upper and lower bounds which minimize the area between the planes under the given domain [l x, u x]. The red bound on (b) grants smaller area, but contains the negative value, which occurs the analysis failure in the audio processing pipeline. i.e., h 2. For this example, we assume the computation in h (t) 2 = 0. Robustness certification using DeepPoly The hidden states in the LSTM at the final timestep are passed through a fully connected layer without activation. In our example we assume that logits after the layer are obtained with the following formula: 2. These are shown in the final box of Fig. 2. To certify that the neural network classifies the input to class 1, we need to prove that l 1 − l 2 > 0. We now apply the back-substitution technique as in Singh et al. (2019a) upto the input to the LSTM: As this value is greater than 0, robustness is established. The process above, of replacing a variable with its constraints, is called back-substitution. Here, we replaced l 1 with h 1 − 0.33 and so on. In Appendix C we show more detailed calculations to obtain tighter bound 0.0375 by also back-substituting in the preprocessing pipeline. For our experiments in Section 6, we tune the number of back-substitution steps to achieve a good tradeoff between speed and precision. Note that robustness cannot be proved if one concretizes the expression above to an interval, instead of performing back-substitution of linear bounds. For instance, if we concretize h 2 + 0.24 ≥ −0.01 which is imprecise and fails to certify robustness. Note that if t is not the last timestep, the hidden state and the cell state are passed to the next analysis timestep instead of computing the final output. As illustrated earlier, the first part of the verification process involves handling of the audio processing stage (performed using MFCC). Here, most matrix multiplication parts can be captured exactly by our abstraction, but MFCC also includes the square operation in the FFT and the logarithm operation in the computation of energy from filterbanks. Thus, to handle these non-linear operations, we need to create new abstract transformers, which we present next. To ensure a minimal loss of precision, our abstract transformers minimize the area between the lower and upper bounds in the input-output plane. This approach of minimizing the area has been used before for other transformers and has been shown to be practically effective in Singh et al. (2018; 2019a). We denote the set of all variables in the analysis as X. For an element x ∈ X, we denote the functions corresponding to its linear lower and upper bound as x l and x u, respectively. These are scalar functions defined on R k where k is the number of other variables used to compute x. For every element, we also maintain interval bounds, x ∈ [l x, u x]. For ease of explanation, we introduce the log transformer followed by the square transformer. Log abstract transformer The logarithm operation is an assignment y:= log(x) where x, y ∈ X. The output of this operation cannot be captured exactly and we need to carefully introduce an approximation. Here, we first compute the minimum l y = log(l x) and the maximum u y = log(u x), which we use as interval approximation. We define linear lower and upper bound functions y l, y u: R → R. Using concavity of the logarithm on any subset of the domain, the lower bound function is a line connecting the points (l x, log(l x)) and (u x, log(u x)). The upper bound function is chosen as the tangent line to the function minimizing the area between the lower and the upper bound. As a , we obtain the following bounds as depicted in Fig. 3a: Note that if l x ≤ 0, y l (x) = −∞ since the function is not defined on that domain. We make an exception when u x − l x < 10 −4 to use interval bound for y l for avoiding the unstable floating point calculation caused by large denominator of x coefficient. Square abstract transformer The square operation is an assignment y:= x 2 where x, y ∈ X. Similar to logarithm, this operation is non-linear and cannot be captured exactly in the abstraction. We first compute the interval bounds of the output l y and u y, and set the minimum value l y to 0 when Next, we define linear lower and upper bound functions y l, y u: R → R. Using the convexity of the square function, we set the upper bound function y u to the linear function connecting (l x, l 2 x) and (u x, u 2 x). For the lower bound function y l, we have to be delicate: since x 2: R → R ≥0, our y l should also be greater or equal than 0 within any domain. With the principle of minimizing the area between the bounds, we obtain the following bounds for cases as shown in Fig. 3b: Most prior work on verification of neural networks focuses on feed-forward or convolutional architectures whose operations consist of a sequence of affine transforms and activation functions. However, to perform verification of recurrent architectures, one needs to handle the updates of the recurrent unit. Following the equations updating the LSTM presented in Section 2, we can observe that pre-activations of gates are affine transforms which can be captured exactly in our abstraction. However, the operations updating the cell and hidden states are non-linear and require approximation. Overall, we have three elementwise products -two are between a sigmoid and a tanh and one between a sigmoid and the identity. A straightforward approach to handling such transforms would be to concretize the polyhedra DeepPoly element to an interval and perform the multiplication using intervals. However, this approach would lose all relations between inputs and incur precision loss in future operations of the analysis. Instead, we design custom binary approximation transformers specifically tailored to handle the elementwise product in the update of the recurrent unit. Sigmoid Tanh abstract transformer We define elementwise multiplication between sigmoid and tanh as an assignment z:= σ(x) tanh(y), where x, y, z ∈ X. As before, our aim is to construct linear bounds z l, z u. Unlike the previously defined DeepPoly transformers which take as input a single abstract element, this transformer is the first which receives two abstract elements as operands. Hence, the bound functions have the form z l, z u: R 2 → R. Our goal is to bound this function between two planes such that the volume between the planes is as small as possible (following our previous heuristic). We first divide the computation into 3 cases based on the signs of l y and u y. To simplify notation, we introduce the variable assignment in Table 1 so that we can reuse notation across all cases. We first define an anchor point a: a point in the box where function f attains the max/min value defined upon the case. Also we need reference point r whereas the plane would meet where ∂y, and Φ, a, r are chosen from Table 1. Finally, we choose bounds from two candidates which minimize the volume between the planes. Fig. 4 is the visualization of this . We note that Sigmoid Identity transformer is handled in the same manner (we can directly apply the same transformer by replacing f (x, y) with σ(x) · y). Theorem 1. Our Sigmoid Tanh transformer is optimal (it minimizes the volume between the lower and upper plane) under the assumptions: bounding planes are tangent to the curve at the respective anchor points, bounding planes are parallel to either x or y axis. Planes computed using our transformer in a volume strictly smaller than those computed using interval analysis. We show the proof in Appendix B. Our assumptions are based on the following reasoning. The first assumption is need so that our transformer produces bounds strictly better than interval analysis. Unless z u passes through the point (u x, u y), the concrete upper bound would be larger than σ(u x) · tanh(u y), making it worse than an interval bound(analogously for the lower bound). The second assumption enables computation of optimal planes in O while without this assumption one needs to solve a non-convex optimization problem which is not feasible at the scale of networks we consider. We now evaluate the effectiveness of DAC on several datasets and neural architectures. All our transformers are implemented in C (for performance) and exposed to the verifier using a Python interface. We will publicly release the datasets, trained networks and source code of DAC. Verification is performed on a Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz using 6 cores for each instance. Experimental setup We evaluate on audio classification benchmarks: Free Spoken Digit Dataset (FSDD) and Google Speech Commands (GSC) . FSDD contains 2,000 samples of spoken digits while GSC is a more complex dataset containing 65,000 utterances of 30 short words spoken by thousands of different people. State-of-the-art accuracy on GSC is 96.9% by de using attention RNN, while our verified model achieves 89%. Following Singh et al. (2019b), we define success rate of verification as the ratio between certified samples and correctly predicted ones. We randomly shuffled the test data and then, for every experiment, inferred labels one by one until the number of correctly classified samples reached 100. We report the number of provably correct samples out of these 100 as our provability. As a baseline, we consider a verification method based on interval analysis, which only derives neuronwise lower and upper bounds using the previous layer's bounds. We used networks with Table 2: Effect of back-substitution depth on the performance. Effect of back-substitution depth In this experiment, we vary the depth of back-substitution used in DAC and study its effect on our performance. We run this experiment on an FSDD network and report the in Table 2. We observe that increasing the back-substitution depth increases the provability. This is due to the fact that we benefit from cancellation of common terms in the propagated expressions, as demonstrated in our example in Section 3. However, this comes at a cost of exponential increase in runtime. Thus, we choose to use depth 3 in order to get high verification rate while still having reasonable verification speed. Provable defense for audio classifiers We also for the first time trained audio classifiers to be provably robust against noise-based perturbations. Our training follows; -we perturb the input signal and propagate interval bounds through the audio processing and LSTM stages. To train, we combine standard loss with the worst case loss obtained using interval propagation. The ing network shown in Fig. 5b achieves 80% of provability for -80 dB even with the imprecise intervals, outperforming the undefended network. Even though this network was specifically trained to be verifiable using intervals, DAC still outperforms intervals and proves significantly more robustness properties. Also note that defended network has lower accuracy of 95%, compared to the 98% accuracy of the baseline. Experimental comparison with prior work POPQORN also proposes a method to certify recurrent neural networks by propagating linear bounds. One of the key differences with our work is the approximation of σ(x) · tanh(y) using linear bounds. We found that, in practice, optimization approach used by POPQORN produces approximations of slightly smaller volume than our LSTM transformer (although non-comparable). However, this smaller volume comes at a high cost in runtime. We tried to integrate POPQORN bounds into our verification framework, however we found it not feasible for our audio tasks as it increased our end-to-end runtime 200× on one of the smaller networks. In contrast, DAC takes only 1-2 minutes on average to verify a single example containing multiple frames. Indeed, our observation is consistent with the evaluation of POPQORN in the paper where it takes hours on a GPU for verifying NLP and vision tasks only for a single frame. We presented the first verifier for certifying audio classifiers. The key idea was to create abstract transformers for non-linear operations used in the audio processing stage and the recurrent network. These transformers compute an optimal (area-wise) approximation under assumptions representable in the underlying convex relaxation and enable sound handling of the entire pipeline. Our evaluation shows that DAC is practically effective and achieves high verification rates on different datasets. by the smaller volume under the each plane. Then for any x, y, f (x, y 1) < f (x, y 2) and f (x 1, y) < f (x 2, y). Thus, since z u x is independent to y, it is sufficient to show z We can easily know that f (x, u y) is concave at x ≥ 0 and convex at x ≤ 0 by the second derivation of f. (a) Consider the case of u x > 0. Let x 0 be the x coordinate of the crossing of f (x, u y) and. Again, by convexity of. Again, by convexity of With analogous steps, z l y can be shown to lie under the curve. Choosing the plane with larger volume underneath it allows to minimize the expected difference between the true curve and the lower bound plane under the randomly chosen domain. The proof of upper bounds will follow the same steps with the first case. z u x in this case is exactly same as before, but since f (x, y) goes below 0 when y < 0, z u y has to anchor at (l x, l y) instead of (u x, l y) since f (l x, l y) ≥ f (u x, l y) and convexity of f in the region. The proof steps do not differ much from the previous proofs. Again, the proof for lower bound is similar as before, but note that z l x needs to choose maximum between the two slopes. This is due to the sign of the values. Since f (u x, l y) < 0 is the minimum in the region and it grows along x gets smaller, both D i f (u x, l y) and f (ux,ly)−f (lx,ly) ux−lx are less than zero. We will not provide the proof since this case is symmetric to the first case. B PROOF OF THEOREM 1 Theorem 1. (copied) Our Sigmoid Tanh transformer is optimal (it minimizes the volume between the lower and upper plane) under the assumptions: bounding planes are tangent to the curve at the respective anchor points, bounding planes are parallel to either x or y axis. Planes computed using our transformer in a volume strictly smaller than those computed using interval analysis. Proof. First note that z We assume the last sound frame s (t) consists of two elements s 1 and s 2 such that:. These constraints capture a noise perturbation of the sound and are depicted in the white box in the left part of Fig. 2. We describe analysis only at the last timestep t (the same process is repeated for every timestep). We note that the DeepPoly abstraction (which we build on in this work) maintains four constraints for every element: a lower and upper constraints which are linear in terms of the previous elements as well as two interval constraints. This abstraction is exact for affine transformations, however, to handle non-linear operations, one has to create new abstract transformers. In our work we introduce such transformers and formally describe their operation in the next sections. In what follows, we show their effect on our running example. For the presentation below, our figure shows the linear constraints obtained by the verifier, but to avoid visual clutter, we do not show the two interval constraints (however, we do list them in the text). The first operation in the audio processing stage is the Fast Fourier Transform (FFT) of the pre-emphasized input signal. It is a two-step process shown in dashed boxes in Fig. 2. The preemphasis is in fact an affine transform so we perform it jointly with the affine transform in the FFT. As the composition of two affine transforms is again affine, this amounts to a single affine transform on the input. In our example, the composed affine transform is y 2. Affine transforms can be captured exactly and here we obtain e.g., s 2. We also obtain the interval bounds y The next step in the computation of FFT is elementwise square of y (t). We denote this operation as 2 ) 2. The square operation is non-linear and cannot be captured exactly which means that we need to decide how to lose precision. Here, we apply our new square abstract transformer (formally defined in Section 4) which provides an optimal linear lower and upper bounds of the square function in terms of area. After applying this transformer, we obtain the bounds: The interval bounds are calculated by: Hence, θ 4, 16]. In practice, FFT is followed by another affine transform which adds together the real and complex component of each element, but we omit this for clarity as it is captured as before, without loss of precision. Filterbanks Transform Our analysis continues with the computation of filter banks of the input, shown in solid boxes in Fig. 2. The first step is an affine transform and in our example we use:ψ 2. Our abstraction is again exact here and additionally computes interval boundsψ 2 ∈. The final step in this transform is the elementwise logarithm of the input. We denote the operation as ψ 2 ). As this is again a non-linear operation, we apply our new log transformer so to obtain: 0.0783ψ The interval bounds are calculated by: 2 ) − 4) + (−6(2s (t) 2 ) − 9) + 0.8637 = 0.6264s Discrete Cosine Transform and ReLU After the Filterbanks Transform, the input is passed through the Discrete Cosine Transform (DCT) followed by a Lifting operation. The analysis of these steps is then provided as an input to a fully connected (FC) layer followed by a ReLU activation. To ease presentation, in our example we combine DCT, Lifting and FC layer in a single affine transform: 2. We show this transform in a dotted box in Fig. 2. This is again captured exactly in our abstraction along with interval bounds x 0.1708, 12.8321]. The affine transform is followed by a ReLU activation x 2 ). In general, we use the ReLU transformer defined in Singh et al. (2019a), which for this example produces: LSTM analysis After the verifier completes the audio processing stage, its output is passed as input to the LSTM cell. This cell also receives a hidden state and a cell state from the previous timestep (shown as a blue box in Fig. 2). In our example, we assume these are given as: h ∈ [0.9, 1]. We note that these elements usually have different interval bounds, but for simplicity in our example we use the same intervals. We focus on the first neuron in the LSTM and compute pre-activations in our example for the forget gate f In order to update the cell state, we need to compute: c 1 ). The left and right summands are computed using our new binary abstract transformers for σ(x) · y and σ(x) · tanh(y) expressions, respectively. Applying our abstract transformers on the summands produces the following bounds: 0.1891f and get c (t) that the bounds of h (t) and c (t−1) are given constant. In practice, we recursively apply the process for those vectors to express the bounds in terms of The next hidden state is computed as h 1 ). We apply our abstract transformer for the σ(x) · tanh(y) expression, we obtain: 2. In this example we will assume h (t) 2 = 0 concretely. Robustness certification using DeepPoly The hidden states in the LSTM at the final timestep are passed through a fully connected layer without activation. In our example we assume that logits after the layer are obtained with the following formula: 2. These are shown in the final box of Fig. 2. To certify that the neural network classifies the input to class 1, we need to prove that l 1 − l 2 > 0. We now apply the same back-substitution technique as Singh et al. (2019a): For the purpose of demonstration, we stop here and plug in previously obtained interval bounds for ψ (t). As this value is greater than 0, robustness is established. Then the cell state and hidden state computations can be rewritten as Comparability of different bounding methods We say that methods A and B are pairwise noncomparable if there exists an input for which method A produces tighter bound than method B, and vice versa. Given this definition, POPQORN is non-comparable with our method. To demonstrate this, in Fig. 6 we show a case where this behavior is manifested. Here, for y ≤ −1 POPQORN (shown as orange plane) produces tighter bound than DAC (shown as blue plane). However, for the other entire range of inputs where −10 ≤ y ≤ −2, POPQORN bounds are substantially worse than our bounds. Further, those bounds are even worse than interval bounds and the overapproximation error is not bounded. Contrary to POPQORN, our bounds are always strictly better than interval bounds (this is proven in Theorem 1) and distance between the function and our planes is bounded. Comparison on synthetic test cases In this experiment, we compare bounds produced by POPQORN and DAC on a set of synthetic test cases, unrelated to the certification of audio classifiers. Both methods take l x, u x and l y, u y as bounds for the input to the sigmoid and tanh, respectively. We sample those inputs uniformly from [−2, 2] × [−2, 2] and compared the volume between the curve σ(x) · tanh(y) and bounding planes produced by both DAC and POPQORN. The volume was computed using 1000 Monde Carlo samples in the set Since there are two σ(x) · tanh(y) and one σ(x) · y calculation appearing in a single LSTM cell, we run the experiment with such portion of data. In other words, in 67% of experiments we bound σ(x) · tanh(y), and in 33% we bound σ(x) · y. We sampled 100 such inputs and compared the volumes obtained by POPQORN and DAC with the volume obtained using interval. The distribution of volumes is shown in Fig. 7. Here, 1 stands for the same volume as interval bounds and values less than 1 indicate performance better than intervals. We conclude that, for this experiment, POPQORN bounds produces smaller volume than our method -0.2 compared to 0.37. In terms of runtime, POPQORN takes 14.37 seconds on average while the bound calculation of DAC finishes in few milliseconds. Direct comparison on test cases from audio benchmarks The previous experiment may not reflect the actual performance on audio benchmarks that we consider in this work. We uniformly sampled the arguments l x, u x, l y, u y that our transformer for σ(x) · tanh(y) and σ(x) · y was invoked with. We distinguish between test cases corresponding to different perturbation ranges and then further split it into certified and uncertified samples. For each of the cases, Fig. 8 shows the box plot of the distributions of volumes produced by both POPQORN and DAC. We found that, while overall POPQORN bounds work well in practice, they frequently produce bounds less precise than interval bounds. The reason of this malfunctioning comes from the limitation of gradient descent approach employed by POPQORN. The gradient of the function, which POPQORN uses to search for the optimal bounds, is large when the inputs are distributed near 0. However, if the inputs are far from the origin or are too close to each other, the function curve becomes almost flat, gradients are close to zero and gradient descent has problems converging. Also, in other cases, gradient descent is not guaranteed to find the bounds with minimum volume. Fig. 6 shows one of the examples where POPQORN fails to find lower planar bound which produces minimum volume. On the contrary, the ing value of σ(x) · tanh(y) is within [−1, 1] regardless of the magnitude of arguments which means that the error produced by intervals is bounded. As our planar bounds are strictly better than intervals, regardless of input conditions, our error is also bounded. Plugging in POPQORN bounds into DAC We also experimented with using POPQORN bounds instead of our bounds in the pipeline and compared the final adversarial regions with those ing from DAC. As POPQORN is relatively slow (108 minutes per sample), we performed this experiment only on the first 10 inputs with −80dB perturbation. Using their bounds in 0 verified samples, while DAC verifies 4 samples. We believe the reason here is the existence of many pathological cases as described in the previous point where gradient descent used by POPQORN converges to suboptimal solution which ends up being worse than interval bounds. These errors are further propagated through each frame and ing output can not be certified. In our experiments, we followed the convention of provability measurement from Singh et al. (2019b). Here, we also provide the with error bars from 10 independent repetitions with randomly permuted test set. For each repetition, we randomly permute the test set with the different seed and collect the first 100 samples with the correct prediction under zero perturbation from the ordered set. We then count the number of certified inputs from those samples to represent the provability under the given constant. Fig. 9 shows that provabilities do not differ much from the reported for multiple experiments. Here we give more details on our training procedure for provably defended network which follows;. Let z LB and z U B be the ing lower and upper bounds for the final logit z ∈ R d under the perturbation size, with the true label j ∈ {0, · · ·, d − 1}. We define the worst-case logitsẑ aŝ i = j which corresponds to the worst concrete logits under the given input with the predefined perturbation amount. Recall that we say the input is certified when this worst-case logits satisfy j = arg max iẑi. Training loss L is a linear combination of the standard cross-entropy loss l(z, e (j) ) and the worst-case cross-entropy loss l(ẑ, e (j) ) where e (j) is target one-hot vector, i.e., L(t) = κ(t)l(z, e (j) ) + (1 − κ(t))l(ẑ( (t)), e (j) ) Note that we set up the κ and as the function of t. As in, gradual increase/decrease of these parameters during the training was essential to get desired performance. We set these functions with respect to the number of training epochs E as κ(t) = 1 − t/E t < E/2 1/2 otherwise (t) = −70 − (200 − 70) 1−t/E. Also, to track the training speed, we increase one by one t only if we are achieving 85% of standard accuracy and 80% provability at the current setting. The models were built under the E = 60. We also attach the missing of defended GSC with 82% concrete accuracy in Fig. 10. We perform the experiments with the same architecture with different model parameters for FSDD and GSC. Table 3 shows the parameters of the models we use. Both defended and undefended networks share the same parameters for each dataset.

Represent the short summary for finding the corresponding scientific article (long)

We present the first approach to certify robustness of neural networks against noise-based perturbations in the audio domain.

This paper 1) characterizes functions representable by ReLU DNNs, 2) formally studies the benefit of depth in such architectures,  3) gives an algorithm to implement empirical risk minimization to global optimality for two layer ReLU nets.


Question: represent the article for finding a single-sentence tldr.

Question: Since deep neural networks are over-parameterized, they can memorize noisy examples. We address such memorizing issue in the presence of annotation noise. From the fact that deep neural networks cannot generalize neighborhoods of the features acquired via memorization, we hypothesize that noisy examples do not consistently incur small losses on the network under a certain perturbation. Based on this, we propose a novel training method called Learning with Ensemble Consensus (LEC) that prevents overfitting noisy examples by eliminating them using the consensus of an ensemble of perturbed networks. One of the proposed LECs, LTEC outperforms the current state-of-the-art methods on noisy MNIST, CIFAR-10, and CIFAR-100 in an efficient manner. Deep neural networks (DNNs) have shown excellent performance on visual recognition datasets . However, it is difficult to obtain highquality labeled datasets in practice (a). Even worse, DNNs could not generalize the training data in the presence of noisy examples. Therefore, there is an increasing demand for robust training methods. In general, DNNs optimized with SGD first generalize clean examples under label noise. Based on this, recent studies consider examples that incur small losses on the network that does not overfit noisy examples as being clean . However, such small-loss examples may be corrupted, particularly under a high level of noise. Hence, choosing safe examples from the noisy dataset with small-loss criteria may be impractical. To address this, we find the method of screening out noisy examples among small-loss examples by focusing on well-known observations: (i) noisy examples are learned via memorization rather than via generalization and (ii) under a certain perturbation, network predictions for memorized features easily fluctuate, while those for generalized features do not. Based on these two observations, we hypothesize that out of small-loss examples, training losses of noisy examples would increase by injecting certain perturbation to network parameters, while those of clean examples would not. This suggests that examples that consistently incur small losses under multiple perturbations can be considered as being clean. Since this idea comes from an artifact of SGD optimization, it can be applied to any architecture optimized with SGD. In this work, we introduce a method of perturbing parameters to filter noisy examples out of smallloss examples. By embedding the filtering into training, we propose a new robust training scheme termed learning with ensemble consensus (LEC). In LEC, the network is first trained on the entire training set for a while and then trained on the intersection of small-loss examples of the ensemble of perturbed networks. We present three LECs with different perturbations and evaluate their effectiveness on three benchmark datasets with random label noise , open-set noise (b), and semantic noise. The proposed LEC outperforms existing robust training methods by efficiently removing noisy examples from training batches. Generalization of DNNs. Although DNNs are over-parameterized, they have impressive generalization ability . Some studies argue that gradient-based optimization plays an important role in regularizing DNNs (; . show that DNNs optimized with gradient-based methods generalize clean examples in the early stage of training. Since mislabeling reduces the correlation with other training examples, it is likely that noisy examples are learned via memorization. Therefore, we analyze the difference between generalized and memorized features to discriminate clean and noisy examples. Training DNNs with Noisy datasets. Label noise issues can be addressed by reducing negative impact of noisy examples. One direction is to train with a modified loss function based on the noise distribution. Most studies of this direction estimate the noise distribution prior to training as it is not accessible in general (; ; ;). Another direction is to train with modified labels using the current model prediction . Aside from these directions, recent work suggests a method of exploiting small-loss examples (; ; ;) based on the generalization ability of DNNs. However, it is still hard to find clean examples by relying on training losses. This study presents a simple method to overcome such a problem of small-loss criteria. Suppose that % of examples in a dataset D:= D clean ∪ D noisy are noisy. Let S,D,θ denote the set of (100-)% small-loss examples of the network f parameterized by θ out of examples in D. Since it is generally hard to learn only all clean examples especially on the highly corrupted training set, it is problematic to regard all examples in S,D,θ as being clean. To mitigate this, we suggest a simple idea: to find noisy examples among examples in S,D,θ. Since noisy examples are little correlated with other training examples, they are likely to be learned via memorization. However, DNNs cannot generalize neighborhoods of the memorized features. This means that even if training losses of noisy examples are small, they can be easily increased under a certain perturbation δ, i.e., for (x, y) ∈ D noisy, Unlike noisy examples, the network f trained on the entire set D can generalize some clean examples in the early stage of training. Thus, their training losses are consistently small in the presence of the perturbation δ, i.e., for (x, y) ∈ D clean, This suggests that noisy examples can be identified from the inconsistency of losses under certain perturbation δ. Based on this, we regard examples in the intersection of (100-)% small-loss examples of an ensemble of M networks generated by adding perturbations δ 1, δ 2,..., δ M to θ, i.e., as being clean. We call it ensemble consensus filtering because examples are selected via ensemble consensus. With this filtering, we propose a new robust training method termed learning with ensemble consensus (LEC) described in Algorithms 1 and 2. Both algorithms consist of warmingup and filtering processes. The difference between these two lies in the filtering process. During the filtering process of Algorithm 1, the network is trained on the intersection of (100-)% small-loss examples of M networks within a mini batch B, thus the number of examples updated at once is changing. We can encourage more stable training with a fixed number of examples to be updated at once as described in Algorithm 2. During the filtering process of Algorithm 2, we first obtain the intersection of small-loss examples of M networks within a full batch D at each epoch. We then sample a subset of batchsize from the intersection and train them at each update like a normal SGD. Require: noisy dataset D with noise ratio %, duration of warmingup Tw, # of networks used for filtering M, perturbation δ 1: Initialize θ randomly 2: for epoch t = 1: Tw do Warming-up process 3: for mini-batch index b = 1: Sample a subset of batchsize B b from a full batch D 5: Ensemble consensus filtering 16: for mini-batch index b = 1: Sample a subset of batchsize B b from a full batch D 5: Ensemble consensus filtering 14: for mini-batch index b = 1: Sample a subset of batchsize B b from D t 16: end for 18: end for Now the goal is to find a perturbation δ to be injected to distinguish between generalized and memorized features. We present three LECs with different perturbations in the following. The pseudocodes can be found in Section A.1.3. • Network-Ensemble Consensus (LNEC): Inspired by the observation that an ensemble of networks with the same architecture is correlated during generalization and is decorrelated during memorization , the perturbation δ comes from the difference between M networks. During the warming-up process, M networks are trained independently. During the filtering process, M networks are trained on the intersection of (100-)% small-loss examples of M networks. • Self-Ensemble Consensus (LSEC): We focus on the relationship between and: network predictions for memorized features are uncertain and those for generalized features are certain. Since the uncertainty of predictions also can be captured by multiple stochastic predictions , the perturbation δ comes from the difference between M stochastic predictions of a single network. 1 During the filtering process, the network is trained on the intersection of (100-)% small-loss examples obtained with M stochastic predictions. • Temporal-Ensemble Consensus (LTEC): Inspired by the observation that during training, atypical features are more easily forgetful compared to typical features , the perturbation δ comes from the difference between networks at current and preceding epochs. During the filtering process, the network is trained on the intersection of (100-)% small-loss examples at the current epoch t and preceding min(M − 1, t − 1) epochs. We collect (100-)% small-loss examples at the preceding epochs, rather than network parameters to reduce memory usage. In this section, we show (i) the effectiveness of three perturbations at removing noisy examples from small-loss examples and (ii) the comparison of LEC and other existing methods under various annotation noises. Annotation noise. We study random label noise , open-set noise (b), and semantic noise. To generate these noises, we use MNIST , CIFAR-10/100 that are commonly used to assess the robustness. For each benchmark dataset, we only corrupt its training set, while leaving its test set intact for testing. The details can be found in Section A.1.1. • Random label noise. Annotation issues can happen in easy images as well as hard images (a). This is simulated in two ways: sym-% and asym-%. For sym-%, % of the entire set are randomly mislabeled to one of the other labels and for asym-%, each label i of % of the entire set is changed to i + 1. We study four types: sym-20% and asym-20% to simulate a low level of noise, and sym-60% and asym-40% to simulate a high level of noise. • Open-set noise. In reality, annotated datasets may contain out-of-distribution (OOD) examples. As in , to make OOD examples, images of % examples randomly sampled from the original dataset are replaced with images from another dataset, while labels are left intact. SVHN is used to make open-set noise of CIFAR-100, and ImageNet-32 and CIFAR-100 are used to make open-set noise of CIFAR-10. We study two types: 20% and 40% open-set noise. • Semantic noise. In general, images with easy patterns are correctly labeled, while images with ambiguous patterns are obscurely mislabeled. To simulate this, we select the top % most uncertain images and then flip their labels to the confusing ones. The uncertainty of each image is computed by the amount of disagreement between predictions of networks trained with clean dataset as in. 2 Then, the label of each image is assigned to the label with the highest value of averaged softmax outputs of the networks trained with a clean dataset except for its ground-truth label. We study two types: 20% and 40% semantic noise. Architecture and optimization. Unless otherwise specified, we use a variant of 9-convolutional layer architecture . All parameters are trained for 200 epochs with Adam with a batch size of 128. The details can be found in Section A.1.2. Hyperparameter. The proposed LEC involves three hyperparameters: duration of warming-up T w, noise ratio %, and the number of networks used for filtering M. Unless otherwise specified, T w is set to 10, and M is set to 5 for random label noise and open-set noise, and 10 for semantic noise. We assume that a noise ratio of % is given. Further study can be found in Section 5.2. Evaluation. We use two metrics: test accuracy and label precision . At the end of each epoch, test accuracy is measured as the ratio of correctly predicted test examples to all test examples, and label precision is measured as the ratio of clean examples used for training to examples used for training. Thus, for both metrics, higher is better. For methods with multiple networks, the averaged values are reported. We report peak as well as final accuracy because a small validation set may be available in reality. For each noise type, every method is run four times with four random seeds, e.g., four runs of Standard on CIFAR-10 with sym-20%. A noisy dataset is randomly generated and initial network parameters are randomized for each run of both random label noise and open-set noise. Note that four noisy datasets generated in four runs are the same for all methods. On the other hand, semantic noise is generated in a deterministic way. Thus, only initial network parameters are randomized for each run of semantic noise. 2 The uncertainty of image x is defined by f (x; θn)) where f (; θ) denotes softmax output of network parameterized by θ. Here, N is set to 5 as in. CIFAR-10, asym-40% Figure 2: Label precision (%) of small-loss examples of the current network (in green) and the intersection of small-loss examples of the current and preceding networks (in red) during running LTEC on CIFAR-10 with random label noise. We report the precision from epoch 11 when the filtering process starts. Comparison with Self-training. In Section 3.1, we argue that (100-)% small-loss examples may be corrupted. To show this, we run LEC with M = 1, which is a method of training on (100-)% small-loss examples. Note that this method is similar to the idea of;. We call it Self-training for simplicity. Figure 1 shows the label precision of Selftraining is low especially under the high level of noise, i.e., sym-60%. Compared to Self-training, three LECs are trained on higher precision data, achieving higher test accuracy as shown in Table 1. Out of these three, LTEC performs the best in both label precision and test accuracy. Noisy examples are removed through ensemble consensus filtering. In LTEC, at every batch update, we first obtain (100-)% small-loss examples of the current network and then train on the intersection of small-loss examples of the current and preceding networks. We plot label precisions of small-loss examples of the current network (in green) and the intersection (in red) during running LTEC on CIFAR-10 with random noise in Figure 2. We observe that label precision of the intersection is always higher, indicating that noisy examples are removed through ensemble consensus filtering. Competing methods. The competing methods include a regular training method: Standard, a method of training with corrected labels: D2L , a method of training with modified loss function based on the noise distribution: Forward , and a method of exploiting small-loss examples: Co-teaching . We tune all the methods individually as described in Section A.1.4. Results on MNIST/CIFAR with random label noise. The overall can be found in Figures 3 and 4, and Table 2. We plot the average as a solid line and the standard deviation as a shadow around the line. Figure 3 states that the test accuracy of D2L increases at the low level of label noise as training progresses, but it does not increase at the high level of label noise. This is because D2L puts large weights on given labels in the early stage of training even under the high level of noise. Forward shows its strength only in limited scenarios such as MNIST. Co-teaching does not work well on CIFAR-100 with asym-40%, indicating that its cross-training scheme is vulnerable to smallloss examples of a low label precision (see Figure 4). Unlike Co-teaching, our methods attempt to remove noisy examples in small-loss examples. Thus, on CIFAR-100 with asym-40% noise, both LTEC and LTEC-full surpass Co-teaching by a wide margin of about 6% and 5%, respectively. Results on CIFAR with open-set noise. The overall can be found in Table 3. All the methods including LTEC and LTEC-full perform well under open-set noise. We speculate that this is due to a low correlation between open-set noisy examples. This is supported by the on CIFAR-10, i.e., all the methods perform better on ImageNet-32 noise than on CIFAR-100 noise, as ImageNet-32 has more classes than CIFAR-100. Similar to poorly annotated examples, out-ofdistribution examples are difficult to be generalized during the warming-up process. Therefore, they can be removed from training batches through ensemble consensus filtering. Results on CIFAR with semantic noise. The overall can be found in Table 4. The semantically generated noisy examples are highly correlated with each other, making it difficult to filter out those examples through ensemble consensus. We use 10 as the value of M for semantic noise because ensemble consensus with a bigger M is more conservative. On CIFAR with semantic noise, LTEC and LTEC-full perform comparably or best, compared to the other methods. Of the two, LTEC-full performs better on 40% semantic noise due to its training stability. It is hard to learn all clean examples during the warming-up process. Therefore, clean examples with large losses may be excluded from training batches during the filtering process. However, we expect that the number of clean examples used for training would increase gradually as training proceeds since LEC allows the network to generalize clean examples without overfitting. To confirm this, we measure recall defined by the ratio of clean examples used for training to all clean examples at the end of each epoch during running LTEC and LTEC-full. As expected, recalls of both LTEC and LTEC-full sharply increase in the first 50 epochs as described in Figure 5. Pre-training prior to the filtering process may help to prevent the removal of clean examples from training batches. The number of networks used for filtering. During the filtering process of LEC, we use only the intersection of small-loss examples of M perturbed networks for training. This means that the number of examples used for training highly depends on M. To understand the effect of M, we run LTEC with varying M on CIFAR-10 with random label noise. In particular, the range of M is {1, 3, 5, 7, 9, ∞}. Table 5 shows that a larger M is not always lead to better performance. This is because too many examples may be removed from training batches as M increases. Indeed, the total number of examples used for training is critical for the robustness as claimed in;. Noise ratio. In reality, only a poorly estimated noise ratio may be accessible. To study the effect of poor noise estimates, we run LTEC on CIFAR-10 with random label noise using a bit lower and higher values than the actual noise ratio as in. We also run Co-teaching that requires the noise ratio for comparison. The overall can be found in Table 6. Since it is generally difficult to learn all clean examples, training on small-loss examples selected using the over-estimated ratio (i.e., 1.1) is often helpful in both Co-teaching and LTEC. In contrast, smallloss examples selected using the under-estimated ratio may be highly corrupted. In this case, LTEC is robust to the estimation error of noise ratio, while Co-teaching is not. Such robustness of LTEC against noise estimation error comes from ensemble consensus filtering. Applicability to different architecture. The key idea of LEC is rooted in the difference between generalizaton and memorization, i.e., the ways of learning clean examples and noisy examples in the early SGD optimization. Therefore, we expect that LEC would be applicable to any architecture optimized with SGD. To support this, we run Standard and LTEC with ResNet-20 . The architecture is optimized based on , achieving the final test accuracy of 90.67% on clean CIFAR-10. Here, T w is set to 30 for the optimization details. Table 7 shows LTEC (ResNet) beats Standard (ResNet) in both peak and final accuracies, as expected. This work presents the method of generating and using the ensemble for robust training. We explore three simple perturbation methods to generate the ensemble and then develop the way of identifying noisy examples through ensemble consensus on small-loss examples. Along with growing attention to the use of small-loss examples for robust training, we expect that our ensemble method will be useful for such training methods. A.1.1 ANNOTATION NOISES • Random label noise: For sym-%, % of the entire set are randomly mislabeled to one of the other labels and for asym-%, each label i of % of the entire set is changed to i + 1. The corruption matrices of sym-% and asym-% are described in Figures A1a and A1b, respectively. • Open-set noise: For % open-set noise, images of % examples randomly sampled from the original dataset are replaced with images from external sources, while labels are left intact. For CIFAR-10 with open-set noise, we sample images from 75 classes of CIFAR-100 and 748 classes of ImageNet to avoid sampling similar images with CIFAR-10. • Semantic noise: For semantic noise, we choose uncertain images and then mislabel them ambiguously. In Figure A2, we see that clean examples have simple and easy images, while noisy examples have not. Also, its corruption matrix (see Figure A1c) describes the similarity between classes, e.g., cat and dog, car and truck, etc. The 9-convolutional layer architecture used in this study can be found in Table A1. The network is optimized with Adam with a batchsize of 128 for 200 epochs. The initial learning rate α is set to 0.1. The learning rate is linearly annealed to zero during the last 120 epochs for MNIST and CIFAR-10, and during the last 100 epochs for CIFAR-100. The momentum parameters β 1 and β 2 are set to 0.9 and 0.999, respectively. β 1 is linearly annealed to 0.1 during the last 120 epochs for MNIST and CIFAR-10, and during the last 100 epochs for CIFAR-100. The images of CIFAR are divided by 255 and are whitened with ZCA. Additional regularizations such as data augmentation are not applied. The on clean MNIST, CIFAR-10, and CIFAR-100 can be found in Table A2. for mini-batch index b = 1: Sample a subset of batchsize B b from a full batch D S,B b,θ:= (100 −)% small-loss examples of f θ within B b Pt ← Pt ∪ S,B b,θ if t < Tw + 1 then Warming-up process 9: 10: else Filtering process 11: if t = 1 then 12: 13: 14: else 16: 19: 5: end for 6: for epoch t = 2: T end do 7: Pt:= (100 −)% small-loss examples of f θ within D Small-loss examples are computed from the 2nd epoch 8: if t < Tw + 1 then Warming-up process 9: for mini-batch index b = 1: Sample a subset of batchsize B b from a full batch D 11: end for else Filtering process if t < M + 1 then 15: else 17: for mini-batch index b = 1: Sample a subset of batchsize B b from D t 21: The competing methods include a regular training method: Standard, a method of training with corrected labels: D2L , a method of training with modified loss function based on the noise distribution: Forward , and a method of exploiting small-loss examples: Co-teaching . We tune all the methods individually as follows: • Standard: The network is trained using the cross-entropy loss. • D2L: The input vector of a fully connected layer in the architecture is used to measure the LID estimates. The parameter involved with identifying the turning point, window size W is set to 12. The network is trained using original labels until the turning point is found and then trained using the bootstrapping target with adaptively tunable mixing coefficient. • Forward: Prior to training, the corruption matrix C where C ji = P(y = i|y true = j) is estimated based on the 97th percentile of probabilities for each class on MNIST and CIFAR-10, and the 100th percentile of probabilities for each class on CIFAR-100 as in. The network is then trained using the corrected labels for 200 epochs. • Co-teaching: Two networks are employed. At every update, they select their small-loss examples within a minibatch and then provide them to each other. The ratio of selected examples based on training losses is linearly annealed from 100% to (100-)% over the first 10 epochs. We compute space complexity as the number of network parameters and computational complexity as the number of forward and backward passes. Here we assume that early stopping is not used and the noise ratio of % is given. Note that the computational complexity of each method depends on its hyperparameter values, e.g., the duration of the warming-up process T w and the noise ratio %. The analysis is reported in Table A3. Our proposed LTEC is the most efficient because it can be implemented with a single network based on Section A.1.3 and only a subset of the entire training set is updated after the warming-up process. Computational complexity # of forward passes n n 2n M n M n n # of backward passes n ≤ n ≤ 2n ≤ M n ≤ n ≤ n A.3.1 OF LTEC WITH M = ∞ Figure A3 shows that ensemble consensus filtering with too large M removes clean examples from training batches in the early stage of the filtering process. Unlike LTEC with M = 5, the recall of LTEC with M = ∞ does not increase as training proceeds, suggesting that its generalization performance is not enhanced. This shows that a larger M does not always lead to better performance. We expect that pre-training prior to the filtering process helps to reduce the number of clean examples removed by ensemble consensus filtering regardless of M. Figure A3: Recall (%) of LTECs with varying M on CIFAR-10 with random label noise.

represent the short summary for finding the corresponding scientific article (long).

This work presents a method of generating and using ensembles effectively to identify noisy examples in the presence of annotation noise.

Represent the short summary for finding the corresponding scientific article (long)

We propose a hypothesis for why gradient descent generalizes based on how per-example gradients interact with each other.