Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Empirical evidence suggests that neural networks with ReLU activations generalize better with over-parameterization. However, there is currently no theoretical analysis that explains this observation. In this work, we study a simplified learning task with over-parameterized convolutional networks that empirically exhibits the same qualitative phenomenon. For this setting, we provide a theoretical analysis of the optimization and generalization performance of gradient descent. Specifically, we prove data-dependent sample complexity bounds which show that over-parameterization improves the generalization performance of gradient descent. Most successful deep learning models use a number of parameters that is larger than the number of parameters that are needed to get zero-training error. This is typically referred to as overparameterization. Indeed, it can be argued that over-parameterization is one of the key techniques that has led to the remarkable success of neural networks. However, there is still no theoretical account for its effectiveness. One very intriguing observation in this context is that over-parameterized networks with ReLU activations, which are trained with gradient based methods, often exhibit better generalization error than smaller networks BID11 ). This somewhat counterintuitive observation suggests that first-order methods which are trained on over-parameterized networks have an inductive bias towards solutions with better generalization performance. Understanding this inductive bias is a necessary step towards a full understanding of neural networks in practice. Providing theoretical guarantees for this phenomenon is extremely challenging due to two main reasons. First, to show a generalization gap, one needs to prove that large networks have better sample complexity than smaller ones. However, current generalization bounds that are based on complexity measures do not offer such guarantees. Second, analyzing the dynamics of first-order methods on networks with ReLU activations is a major challenge. Indeed, there do not exist optimization guarantees even for simple learning tasks such as the classic XOR problem in two dimensions. 1 To advance this issue, we focus on a particular learning setting that captures key properties of the over-parameterization phenomenon. We consider a high-dimensional extension of the XOR problem, which we refer to as the "XOR Detection problem (XORD)". The XORD is a pattern recognition task where the goal is to learn a function which classifies binary vectors according to whether they contain a two-dimensional binary XOR pattern (i.e., or (−1, −1)). This problem contains the classic XOR problem as a special case when the vectors are two dimensional. We consider learning this function with gradient descent trained on an over-parameterized convolutional neural network (i.e., with multiple channels) with ReLU activations and three layers: convolutional, max pooling and fully connected. As can be seen in FIG0, over-parameterization improves generalization in this problem as well. Therefore it serves as a good test-bed for understanding the role of over-parameterization. 1 We are referring to the problem of learning the XOR function given four two-dimensional points with binary entries, using a moderate size one-hidden layer neural network (e.g., with 50 hidden neurons). Note that there are no optimization guarantees for this setting. Variants of XOR have been studied in BID10; but these works only analyzed the optimization landscape and did not provide guarantees for optimization methods. We provide guarantees for this problem in Sec. 9. 3). The figure shows the test error obtained for different number of channels k. The blue curve shows test error when restricting to cases where training error was zero. It can be seen that increasing the number of channels improves the generalization performance. Experimental details are provided in Section 8.2.1.. In this work we provide an analysis of optimization and generalization of gradient descent for XORD. We show that for various input distributions, ranges of accuracy and confidence parameters, sufficiently over-parameterized networks have better sample complexity than a small network which can realize the ground truth classifier. To the best of our knowledge, this is the first example which shows that over-paramaterization can provably improve generalization for a neural network with ReLU activations. Our analysis provides a clear distinction between the inductive bias of gradient descent for overparameterized and small networks. It reveals that over-parameterized networks are biased towards global minima that detect more patterns in the data than global minima found by small networks. 2 Thus, even though both networks succeed in optimization, the larger one has better generalization performance. We provide experiments which show that the same phenomenon occurs in a more general setting with more patterns in the data and non-binary input. We further show that our analysis can predict the behavior of over-parameterized networks trained on MNIST and guide a compression scheme for over-parameterized networks with a mild loss in accuracy (Sec. 6). In recent years there have been many works on theoretical aspects of deep learning. We will refer to those that are most relevant to this work. First, we note that we are not aware of any work that shows that generalization performance provably improves with over-parameterization. This distinguishes our work from all previous works. Several works study convolutional networks with ReLU activations and their properties BID4 b; BID0. All of these works consider convolutional networks with a single channel. BID2 and BID7 provide guarantees for SGD in general settings. However, their analysis holds for over-parameterized networks with an extremely large number of neurons that are not used in practice (e.g., the number of neurons is a very large polynomial of certain problem parameters). Furthermore, we consider a 3-layer convolutional network with max-pooling which is not studied in these works. , BID3 and study the role of overparameterization in the case of quadratic activation functions. BID1 provide generalization guarantees for over-parameterized networks with Leaky ReLU activations on linearly separable data. prove generalization bounds for neural networks. However, these bounds are empirically vacuous for over-parameterized networks and they do not prove that networks found by optimization algorithms give low generalization bounds. We begin with some notations and definitions. Let d ≥ 4 be an integer. We consider a classification problem in the space {±1} 2d. Namely, the space of vectors of 2d coordinates where each coordinate can be +1 or −1. Given a vector x ∈ {±1} 2d, we consider its partition into d sets of two coordinates as follows x = (x 1, ..., x d) where x i ∈ {±1}2. We refer to each such x i as a pattern in x. Neural Architecture: We consider learning with the following three-layer neural net model. The first layer is a convolutional layer with non-overlapping filters and multiple channels, the second layer is max pooling and the third layer is a fully connected layer with 2k hidden neurons and weights fixed to values ±1. Formally, for an input x = (x 1, ..., x d) ∈ R 2d where x i ∈ R 2, the output of the network is given by: DISPLAYFORM0 where W ∈ R 2k×2 is the weight matrix whose rows are the w (i) vectors followed by the u DISPLAYFORM1 vectors, and σ(x) = max{0, x} is the ReLU activation applied element-wise. See FIG6 for an illustration of this architecture. Remark 3.1. Because there are only 4 different patterns, the network is limited in terms of the number of different rules it can implement. Specifically, it is easy to show that its VC dimension is at most 15 (see Sec. 10). Despite this limited expressive power, there is a generalization gap between small and large networks in this setting, as can be seen in FIG0, and in our analysis below. Data Generating Distribution: Next we define the classification rule we will focus on. Let P XOR correspond to the following two patterns: P XOR = {, (−1, −1)}. Define the classification rule: DISPLAYFORM2 Namely, f * detects whether a pattern in P XOR appears in the input. In what follows, we refer to P XOR as the set of positive patterns and {±1} 2 \ P XOR as the set of negative patterns. Let D be a distribution over X × {±1} such that for all (x, y) ∼ D we have y = f * (x). We say that a point (x, y) is positive if y = 1 and negative otherwise. Let D + be the marginal distribution over {±1} 2d of positive points and D − be the marginal distribution of negative points. In the following definition we introduce the notion of diverse points, which will play a key role in our analysis. Definition 3.2 (Diverse Points). We say that a positive point (x, 1) is diverse if for all z ∈ {±1} 2 there exists 1 ≤ i ≤ d such that x i = z. We say that a negative point DISPLAYFORM3 For φ ∈ {−, +} define p φ to be the probability that x is diverse with respect to D φ. For example, if both D + and D − are uniform, then by the inclusion-exclusion principle it follows that DISPLAYFORM4 For each set of binary patterns A ⊆ {±1} 2 define p A to be the probability to sample a point which contains all patterns in A and no patterns in A c (the complement of A). Let A 1 = {2}, A 2 = {4}, A 3 = {2, 4, 1} and A 4 = {2, 4, 3}.The following quantity will be useful in our analysis: DISPLAYFORM5 Learning Setup: Our analysis will focus on the problem of learning f * from training data with a three layer neural net model. The learning algorithm will be gradient descent, randomly initialized. As in any learning task in practice, f * is unknown to the training algorithm. Our goal is to analyze the performance of gradient descent when given data that is labeled with f *. We assume that we are given a training set S = S + ∪ S − ⊆ {±1} 2d × {±1} 2 where S + consists of m IID points drawn from D + and S − consists of m IID points drawn from D −. Importantly, we note that the function f * can be realized by the above network with k = 2. Indeed, the network N defined by the filters w =, DISPLAYFORM6 2d. It can be seen that for k = 1, f * cannot be realized. Therefore, any k > 2 is an over-parameterized setting. Training Algorithm: We will use gradient descent to optimize the following hinge-loss function. DISPLAYFORM7 for γ ≥ 1. 4 We assume that gradient descent runs with a constant learning rate η and the weights are randomly initiliazed with IID Gaussian weights with mean 0 and standard deviation σ g. Furthermore, only the weights of the first layer, the convolutional filters, are trained. We will need the following notation. Let W t be the weight matrix in iteration t of gradient descent. DISPLAYFORM0 2 the ith convolutional filter at iteration t. Similarly, for DISPLAYFORM1 t ∈ R 2 to be the k + i convolutional filter at iteration t. We assume that each DISPLAYFORM2 0 is initialized as a Gaussian random variable where the entries are IID and distributed as N (0, σ 2 g). In each iteration, gradient descent performs the update DISPLAYFORM3 In this section we state our main that demonstrates the generalization gap between overparameterized networks and networks with k = 2. Define the generalization error to be the difference between the 0-1 test error and the 0-1 training error. For any, δ and training algorithm let m(, δ) be the sample complexity of a training algorithm, namely, the number of minimal samples the algorithm needs to get at most generalization error with probability at least 1 − δ. We consider running gradient descent in two cases, when k ≥ 120 and k = 2. In the next section we exactly define under which set of parameters gradient descent runs, e.g., which constant learning rates. Fix parameters p + and p − of a distribution D and denote by c < 10 −10 a negligible constant. Assume that gradient descent is given a sample of points drawn from D + and D −. We denote the sample complexity of gradient descent in the cases k ≥ 120 and k = 2, by m 1 and m 2, respectively. The following shows a data dependent generalization gap (recall the definition of p * in Eq. 3).Theorem 4.1. Let D be a distribution with paramaters p +, p − and p DISPLAYFORM0 The proof follows from Theorem 5.2 and Theorem 5.3 which we state in the next section. The proof is given in Sec. 8.8. One surprising fact of this theorem is that m 1 (0, δ) ≤ 2. Indeed, our analysis shows that for an over-parameterized network and for sufficiently large p + and p −, one diverse positive point and one diverse negative suffice for gradient descent to learn f * with high probability. We note that even in this case, the dynamics of gradient descent is highly complex. This is due to the randomness of the initialization and to the fact that there are multiple weight filters in the network, each with different dynamics. See Sec. 5 for further details. We will illustrate the guarantee of Theorem 4.1 with several numerical examples. In all of the examples we assume that for the distribution D, the probability to sample a positive point is 1 2 and 4 In practice it is common to set γ to 1. In our analyis we will need γ ≥ 8 to guarantee generalization. In Section 8.3 we show empirically, that for this task, setting γ to be larger than 1 in better test performance than setting γ = 1.5 Note that BID6 show that fixing the last layer weights to ±1 does not degrade performance in various tasks. This assumption also appeared in other works BID1 BID8. 6 We note that this generalization gap holds for global minima (0 train error). Therefore, the theorem can be read as follows. For k ≥ 120, given 2 samples, with probability at least 1 − δ, gradient descent converges to a global minimum with at most test error. On the other hand, for k = 2 and given number of samples less than 2 log DISPLAYFORM1, with probability greater than δ, gradient descent converges to a global minimum with error greater than. See Section 5 for further details. DISPLAYFORM2 (it is easy to constuct such distributions). In the first example, we assume that p + = p − = 0.98 and δ = 1 − 0.98 DISPLAYFORM3 05. In this case we get that for any 0 ≤ < 0.005, m 1 (, δ) ≤ 2 whereas m 2 (, δ) ≥ 129. For the second example consider the case where p + = p − = 0.92. It follows that for δ = 0.16 and any 0 ≤ < 0.02 it holds that m 1 (, δ) ≤ 2 and m 2 (, δ) ≥ 17. For = 0 and any δ > 0, by setting p + and p − to be sufficiently close to 1, we can get an arbitrarily large gap between m 1 (, δ) and m 2 (, δ). In contrast, for sufficiently small p + and p −, e.g., in which p +, p − ≤ 0.7, our bound does not guarantee a generalization gap. In this section we sketch the proof of Theorem 4.1. The theorem follows from two theorems: Theorem 5.2 for over-parameterized networks and Theorem 5.3 for networks with k = 2. We formally show this in Sec. 8.8. In Sec. 5.1 we state Theorem 5.2 and outline its proof. In Sec. 5.2 we state Theorem 5.3 and shortly outline its proof. Finally, for completeness, in Sec. 9 we also provide a convergence guarantee for the XOR problem with inputs in {±1}, which in our setting is the case of d = 1. In what follows, we will need the following formal definition for a detection of a pattern by a network. DISPLAYFORM0 We say that a pattern v (positive or negative) is detected by the network N W with confidence DISPLAYFORM1 The above definition captures a desired property of a network, namely, that its filters which are connected with a positive coefficient in the last layer, have high correlation with the positive patterns and analogously for the remaining filters and negative patterns. We note however, that the condition in which a network detects all patterns is not equivalent to realizing the ground truth f *. The former can hold without the latter and vice versa. Theorem 5.2 and Theorem 5.3 together imply a clear characterization of the different inductive biases of gradient descent in the case of small (k = 2) and over-parameterized networks. The characterization is that over-parameterized networks are biased towards global minima that detect all patterns in the data, whereas small networks with k = 2 are biased towards global minima that do not detect all patterns (see Definition 5.1). In Sec. 8.5 we show this empirically in the XORD problem and in a generalization of the XORD problem. In the following sections we will need several notations. Define x 1 =, x 2 = (1, −1), x 3 = (−1, −1), x 4 = (−1, 1) to be the four possible patterns in the data and the following sets: DISPLAYFORM2 We denote by x + a positive diverse point and x − a negative diverse point. Define the following sum: DISPLAYFORM3 Finally, in all of the in this section we will denote by c < 10 −10 a negligible constant. The main in this section is given by the following theorem., k ≥ 120 and γ ≥ 8. Then, with probability DISPLAYFORM0 iterations, it converges to a global minimum which satisfies: DISPLAYFORM1, all patterns are detected with confidence c d.This shows that given a small training set size, and sufficiently large p + and p −, overparameterized networks converge to a global minimum which realizes the classifier f * with high probability and in a constant number of iterations. Furthermore, this global minimum detects all patterns in the data with confidence that increases with over-parameterization. The full proof of Theorem 5.2 is given in Sec. 8.6.We will now sketch its proof. With probability at least (p + p −) m all training points are diverse and we will condition on this event. From Sec. 10 we can assume WLOG that the training set consists of one positive diverse point x + and one negative diverse point x − (since the network will have the same output on all same-label diverse points). We note that empirically over-parameterization improves generalization even when the training set contains non-diverse points (see FIG0 and Sec. 8.2). Now, to understand the dynamics of gradient descent it is crucial to understand the dynamics of the sets in Eq. 5. This follows since the gradient updates are expressed via these sets. Concretely, let DISPLAYFORM2 then the gradient update is given as follows: DISPLAYFORM3 the gradient update is given by: DISPLAYFORM4 Furthermore, the values of N W (x +) and N W (x −) depend on these sets and their corresponding weight vectors, via sums of the form S + t, defined above. The proof consists of a careful analysis of the dynamics of the sets in Eq. 5 and their corresponding weight vectors. For example, one of this analysis is that for all t ≥ 1 and i ∈ {1, 3} we have W There are two key technical observations that we apply in this analysis. First, with a small initialization and with high probability, for all 1 ≤ j ≤ k and 1 ≤ i ≤ 4 it holds that w DISPLAYFORM5. This allows us to keep track of the dynamics of the sets in Eq. 5 more easily. For example, by this observation it follows that if for some j * ∈ W + t it holds that j * ∈ W + t+1, then for all j such that j ∈ W + t it holds that j ∈ W + t+1. Hence, we can reason about the dynamics of several filters all at once, instead of each one separately. Second, by concentration of measure we can estimate the sizes of the sets in Eq. 5 at iteration t = 0. Combining this with of the kind W + t (i) = W + 0 (i) for all t, we can understand the dynamics of these sets throughout the optimization process. The theorem consists of optimization and generalization guarantees. For the optimization guarantee we show that gradient descent converges to a global minimum. To show this, the idea is to characterize the dynamics of S + t using the characterization of the sets in Eq. 5 and their corresponding weight vectors. We show that as long as gradient descent did not converge to a global minimum, S + t cannot decrease in any iteration and it is upper bounded by a constant. Furthermore, we show that there cannot be too many consecutive iterations in which S + t does not increase. Therefore, after sufficiently many iterations gradient descent will converge to a global minimum. We will now outline the proof of the generalization guarantee. Denote the network learned by gradient descent by N W T. First, we show that the network classifies all positive points correctly. Define the following sums for all 1 ≤ i ≤ 4: DISPLAYFORM6 First we notice that for all positive z we have N W T (z) min{X . Hence, we can show that each positive point is classified correctly. The proof that all negative points are classified correctly and patterns x 2 and x 4 are detected is similar but slightly more technical. We refer the reader to Sec. 8.6 for further details. DISPLAYFORM7 The following theorem provides generalization lower bounds of global minima in the case that k = 2 and in a slightly more general setting than the one given in Theorem 5.2. Theorem 5.3. Let S = S + ∪ S − be a training set as in Sec. 3. Assume that gradient descent runs with parameters η = cη k where c η ≤ DISPLAYFORM0, k = 2 and γ ≥ 1. Then the following holds: DISPLAYFORM1 48, gradient descent converges to a global minimum that has non-zero test error. Furthermore, for c d ≥ 2c η, there exists at least one pattern which is not detected by the global minimum with confidence c d .2. The non-zero test error above is at least p * .The theorem shows that for a training set that is not too large and given sufficiently large p + and p −, with constant probability, gradient descent will converge to a global minimum that is not the classifier f * . Furthermore, this global minimum does not detect at least one pattern. The proof of the theorem is given in Sec. 8.7.We will now provide a short outline of the proof. Let w DISPLAYFORM2 T be the filters of the network at the iteration T in which gradient descent converges to a global minimum. The proof shows that gradient descent will not learn f * if one of the following conditions is met: a) DISPLAYFORM3 T · x 4 > 0. Then by using a symmetry argument which is based on the symmetry of the initialization and the training data it can be shown that one of the above conditions is met with high constant probability. Finally, it can be shown that if one of these conditions hold, then at least one pattern is not detected. We perform several experiments that corroborate our theoretical findings. In Sec. 8.5 we empirically demonstrate our insights on the inductive bias of gradient descent. In Sec. 6.2 we evaluate a model compression scheme implied by our , and demonstrate its success on the MNIST dataset. In this section we perform experiments to examine the insights from our analysis on the inductive bias of gradient descent. Namely, that over-parameterized networks are biased towards global minima that detect more patterns in the data than global minima found by smaller networks. We check this both on the XORD problem which contains 4 possible patterns in the data and on an instance of an extension of the XORD problem, that we refer to as the Orthonormal Basis Detection (OBD) problem, which contains 60 patterns in the data. In Sec. 8.5 we provide details on the experimental setups. standard training (red), the small network that uses clusters from the large network (blue), and the large network (120 channels) with standard training (green). It can be seen that the large network is effectively compressed without losing much accuracy. Due to space considerations, we will not formally define the OBD problem in this section. We refer the reader to Sec. 8.5 for a formal definition. Informally, The OBD problem is a natural extension of the XORD problem that contains more possible patterns in the data and allows the dimension of the filters of the convolutional network to be larger. The patterns correspond to a set of orthonormal vectors and their negations. The ground truth classifier in this problem can be realized by a convolutional network with 4 channels. In FIG1 we show experiments which confirm that in the OBD problem as well, overparameterization improves generalization. We further show the number of patterns detected in %0 training error solutions for different number of channels, in both the XORD and OBD problems. It can be clearly seen that for both problems, over-parameterized networks are biased towards %0 training error solutions that detect more patterns, as predicted by the theoretical . By inspecting the proof of Theorem 5.2, one can see that the dynamics of the filters of an overparameterized network are such that they either have low norm, or they have large norm and they point to the direction of one of the patterns (see, e.g., Lemma 8.4 and Lemma 8.6). This suggests that by clustering the filters of a trained over-parameterized network to a small number of clusters, one can create a significantly smaller network which contains all of the detectors that are needed for good generalization performance. Then, by training the last layer of the network, it can converge to a good solution. Following this insight, we tested this procedure on the MNIST data set and a 3 layer convolutional network with convolutional layer with multiple channels and 3 × 3 kernels, max pooling layer and fully connected layer. We trained an over-parameterized network with 120 channels, clustered its filters with k-means into 4 clusters and used the cluster centers as initialization for a small network with 4 channels. Then we trained only the fully connected layer of the small network. In FIG5 we show that for various training set sizes, the performance of the small network improves significantly with the new initialization and nearly matches the performance of the overparameterized network. In this paper we consider a simplified learning task on binary vectors and show that overparameterization can provably improve generalization performance of a 3-layer convolutional network trained with gradient descent. Our analysis reveals that in the XORD problem overparameterized networks are biased towards global minima which detect more relevant patterns in the data. While we prove this only for the XORD problem and under the assumption that the training set contains diverse points, our experiments clearly show that a similar phenomenon occurs in other settings as well. We show that this is the case for XORD with non-diverse points FIG0 ) and in the more general OBD problem which contains 60 patterns in the data and is not restricted to binary inputs FIG1. Furthermore, our experiments on MNIST hint that this is the case in MNIST as well FIG5.By clustering the detected patterns of the large network we could achieve better accuracy with a small network. This suggests that the larger network detects more patterns with gradient descent even though its effective size is close to that of a small network. We believe that these insights and our detailed analysis can guide future work for showing similar in more complex tasks and provide better understanding of this phenomenon. It would also be interesting to further study the implications of such on model compression and on improving training algorithms. Behnam Neyshabur, Zhiyuan Li, Srinadh Bhojanapalli, Yann LeCun, and Nathan Srebro. We tested the generalization performance in the setup of Section3. We considered networks with number of channels 4,6,8,20,50,100 and 200. The distribution in this setting has p + = 0.5 and p − = 0.9 and the training sets are of size 12 (6 positive, 6 negative). Note that in this case the training set contains non-diverse points with high probability. The ground truth network can be realized by a network with 4 channels. For each number of channels we trained a convolutional network 100 times and averaged the . In each run we sampled a new training set and new initialization of the weights according to a gaussian distribution with mean 0 and standard deviation 0.00001. For each number of channels c, we ran gradient descent with learning rate 0.04 c and stopped it if it did not improve the cost for 20 consecutive iterations or if it reached 30000 iterations. The last iteration was taken for the calculations. We plot both average test error over all 100 runs and average test error only over the runs that ended at 0% train error. In this case, for each number of channels 4, 6, 8, 20, 50, 100,200 the number of runs in which gradient descent converged to a 0% train error solution is 62, 79, 94, 100, 100, 100, 100, respectively. Figure 5 shows that setting γ = 5 gives better performance than setting γ = 1 in the XORD problem. The setting is similar to the setting of Section 8.2.1. Each point is an average test error of 100 runs.. Because the is a lower bound, it is desirable to understand the behaviour of gradient descent for values outside these ranges. In Figure 6 we empirically show that for values outside these ranges, there is a generalization gap between gradient descent for k = 2 and gradient descent for larger k. We will first formally define the OBD problem. Fix an even dimension parameter d 1 ≥ 2. In this problem, we assume there is an orthonormal basis B = {v 1, ..., v d1} of R d1. Divide B into two equally sized sets B 1 and B 2, each of size d1 2. Now define the set of positive patterns to be P = {v | v ∈ B 1} ∪ {−v | v ∈ B 1} and negative patterns to be DISPLAYFORM0 we assume the input domain is X ⊆ R d1d2 and each x ∈ X is a vector such that x = (x 1, ..., x d2) where each x i ∈ P OBD. We define the ground truth classifier f OBD: X → {±1} such that f OBD (x) = 1 if and only there exists at least one x i such that x i ∈ P. Notice that for d 1 = 2 and by normalizing the four vectors in {±1} 2 to have unit norm, we get the XORD problem. We note that the positive patterns in the XORD problem are defined to be P XOR and the negative patterns are {±1} 2 \ P XOR.Let D be a distribution over X 2d × {±1} such that for all (x, y) ∼ D, y = f OBD (x). As in the XORD problem we define the distributions D + and D −. We consider the following learning task which is the same as the task for the XORD problem. We assume that we are given a training set S = S + ∪ S − ⊆ {±1} d1d2 × {±1} where S + consists of m IID points drawn from D + and S − consists of m IID points drawn from D −. The goal is to train a neural network with randomly initialized gradient descent on S and obtain a network N: DISPLAYFORM1 We consider the same network as in the XORD problem (Eq. 1), but now the filters of the convolution layer are d 1 -dimensional. Formally, for an input x = (x 1, ..., x d) ∈ X the output of the network is given by DISPLAYFORM2 where W ∈ R 2k×d1 is the weight matrix which contains in the first k rows the vectors w (i) ∈ R d1, in the next k rows the vectors u (i) ∈ R d1 and σ(x) = max{0, x} is the ReLU activation applied element-wise. We performed experiments in the case that d 1 = 30, i.e., in which there are 60 possible patterns. In FIG1, for each number of channels we trained a convolutional network given in Eq. 9 with gradient descent for 100 runs and averaged the . The we sampled 25 positive points and 25 negative points in the following manner. For each positive point we sampled with probability 0.25 one of the numbers twice with replacement. Denote these numbers by m 1 and m 2. Then we sampled m 1 different positive patterns and m 2 different negative patterns. Then we filled a 60d 1 -dimensional vectors with all of these patterns. A similar procedure was used to sample a negative point. We considered networks with number of channels 4,6,8,20,100 and 200 and 500. Note that the ground truth network can be realized by a network with 4 channels. For each number of channels we trained a convolutional network 100 times and averaged the . For each number of channels c, we ran gradient descent with learning rate 0.2 c and stopped it if it did not improve the cost for 20 consecutive iterations or if it had 0% training error for 200 consecutive iterations or if it reached 30000 iterations. The last iteration was taken for the calculations. We plot both average test error over all 100 runs and average test error only over the runs that ended at 0% train error. For each number of channels 4, 6, 8, 20, 100, 200,500 the number of runs in which gradient descent converged to a 0% train error solution is 96, 99, 100, 100, 100, 100, 100, respectively. For each 0% train error solution we recorded the number of patterns detected with c d = 0.0001 according to the Definition 5.1 (generalized to the OBD problem). In the XORD problem we recorded similarly the number of patterns detected in experiments which are identical to the experiments in Section 8.2.1, except that in this case p + = p − = 0.98. We will first need a few notations. Define x 1 =, x 2 = (1, −1), x 3 = (−1, −1), x 4 = (−1, 1) and the following sets: DISPLAYFORM0 We can use these definitions to express more easily the gradient updates. Concretely, let j ∈ W + t (i 1) ∩ W − t (i 2) then the gradient update is given as follows: DISPLAYFORM1 the gradient update is given by: DISPLAYFORM2 We denote by x + a positive diverse point and x − a negative diverse point. Define the following sums for φ ∈ {+, −}: DISPLAYFORM3 By the conditions of the theorem, with probability at least (p + p −) m all the points in the training set are diverse. From now on we will condition on this event. Furthermore, without loss of generality, we can assume that the training set consists of one diverse point x + and one negative points x −. This follows since the network and its gradient have the same value for two different positive diverse points and two different negative points. Therefore, this holds for the loss function defined in Eq. 4 as well. We will now proceed to prove the theorem. In Section 8.6.1 we prove on the filters at initialization. In Section 8.6.2 we prove several auxiliary lemmas. In Section 8.6.3 we prove upper bounds on S − t, P + t and P − t for all iterations t. In Section 8.6.4 we characterize the dynamics of S + t and in Section 8.6.5 we prove an upper bound on it together with upper bounds on N Wt (x +) and −N Wt (x −) for all iterations t. We provide an optimization guarantee for gradient descent in Section 8.6.6. We prove generalization guarantees for the points in the positive class and negative class in Section 8.6.7 and Section 8.6.8, respectively. We complete the proof of the theorem in Section 8.6.9. Lemma 8.1. With probability at least 1 − 4e −8, it holds that DISPLAYFORM0 Proof. Without loss of generality consider DISPLAYFORM1 2, we get by Hoeffding's inequality DISPLAYFORM2 The now follows by the union bound. Lemma 8.2. With probability DISPLAYFORM3 Proof. Let Z be a random variable distributed as N (0, σ 2). Then by Proposition 2.1.2 in , we have DISPLAYFORM4 Therefore, for all 1 ≤ j ≤ k and 1 ≤ i ≤ 4, DISPLAYFORM5 and DISPLAYFORM6 The follows by applying a union bound over all 2k weight vectors and the four points DISPLAYFORM7 From now on we assume that the highly probable event in Lemma 8.2 holds. DISPLAYFORM8 Proof. By Lemma 8.2 we have DISPLAYFORM9 and similarly −N W0 (x −) < 1. Therefore, by Eq. 10 and Eq. 11 we get: DISPLAYFORM10 2. For i ∈ {2, 4} and j ∈ W + 0 (i), it holds that w DISPLAYFORM11 4. For i ∈ {2, 4} and j ∈ U + 0 (i), it holds that u DISPLAYFORM12 2. For i ∈ {2, 4} and j ∈ W + 0 (i), it holds that w DISPLAYFORM13 4. For i ∈ {2, 4} and j ∈ U + 0 (i), it holds that u DISPLAYFORM14 As before, by Lemma 8.2 we have N W2 (x +) < γ and −N W2 (x −) < 1. Lemma 8.4. For all t ≥ 1 we have W DISPLAYFORM0 Proof. We will first prove that DISPLAYFORM1 To prove this, we will show by induction on t ≥ 1, that for all j ∈ W + 0 (i) ∩ W + 0 (l), where l ∈ {2, 4} the following holds: DISPLAYFORM2 The claim holds for t = 1 by the proof of Lemma 8.3. Assume it holds for t = T. By the induction hypothesis there exists an l ∈ {2, 4} such that j ∈ W + T (i) ∩ W − T (l). By Eq. 10 we have, DISPLAYFORM3 where a ∈ {0, 1} and b ∈ {−1, 0}. T ·x l = w (j) 0 ·x l then l = l and either w (j) DISPLAYFORM0 T · x l < 0 and l = l. It follows that either w DISPLAYFORM1 In both cases, we have w DISPLAYFORM2 In order to prove the lemma, it suffices to show that DISPLAYFORM0.., k}. We will show by induction on t ≥ 1, that for all j ∈ W + 0 ∪ W + 0, the following holds: DISPLAYFORM1 The claim holds for t = 1 by the proof of Lemma 8.3. Assume it holds for t = T. By the induction hypothesis j ∈ W + T ∩ W + T. Assume without loss of generality that j ∈ W + T. This implies that j ∈ W − T as well. Therefore, by Eq. 10 we have DISPLAYFORM2 where a ∈ {0, 1} and b ∈ {0, −1}. By the induction hypothesis, w DISPLAYFORM3 where the first inequality follows since j ∈ W + T and the second by Eq. 13. This implies that DISPLAYFORM4 Otherwise, assume that a = 0 and b = −1. By Lemma 8.2 we have w T · x 2 < 0 and j / ∈ W + T, which is a contradiction. DISPLAYFORM5 which concludes the proof. Lemma 8.5. For all t ≥ 0 we have DISPLAYFORM6 0 + α t ηx 2 for α t ∈ Z. This follows since the inequalities u DISPLAYFORM7. Assume by contradiction that there exist an iteration t for which u DISPLAYFORM8 0 + α t−1 ηx 2 where α t−1 ∈ Z. 9 Since the coefficient of x i changed in iteration t, we have j ∈ U + t−1 ∪ U + t−1. However, this contradicts the claim above which shows that if u DISPLAYFORM9 Lemma 8.6. Let i ∈ {1, 3} and l ∈ {2, 4}. DISPLAYFORM10 Proof. First note that by Eq. 11 we generally have u DISPLAYFORM11, by the gradient update in Eq. 11 it holds that a t ∈ {0, −1}. Indeed, a 0 = 0 and by the gradient update if a t−1 = 0 or a t−1 = −1 then a t ∈ {−1, 0}.Assume by contradiction that there exists an iteration t > 0 such that b t = −1 and b t−1 = 0. Note that by Eq. 11 this can only occur if j ∈ U + t−1 (l). We have u DISPLAYFORM12 Lemma 8.7. Let DISPLAYFORM13 and Y DISPLAYFORM14 Then for all t, DISPLAYFORM15 9 Note that in each iteration βt changes by at most η. Proof. We will prove the claim by induction on t. For t = 0 this clearly holds. Assume it holds for t = T. Let j 1 ∈ W + T and j 2 ∈ W + T. By Eq. 10, the gradient updates of the corresponding weight vector are given as follows: DISPLAYFORM16 where a ∈ {0, 1} and b 1, b 2 ∈ {−1, 0, 1}. By Lemma 8.4, j 1 ∈ W + T +1 and j 2 ∈ W + T +1. Therefore, DISPLAYFORM17 DISPLAYFORM18 Proof. In Lemma 8.4 we showed that for all t ≥ 0 and j ∈ W DISPLAYFORM19 t · x 2 ≤ η. This proves the first claim. The second claim follows similarly. Without loss of generality, let j ∈ U + t. By Lemma 8.5 it holds that U DISPLAYFORM20 Therefore, by Lemma 8.6 we have u (j) t x 1 < η, from which the claim follows. For the third claim, without loss of generality, assume by contradiction that for j ∈ U DISPLAYFORM21, from which the claim follows. Lemma 8.9. The following holds: DISPLAYFORM0 Under review as a conference paper at ICLR 2019 DISPLAYFORM1 Proof.1. The equality follows since for each i ∈ {1, 3}, l ∈ {2, 4} and j ∈ W DISPLAYFORM2 2. In this case for each i ∈ {1, 3}, l ∈ {2, 4} and j ∈ W DISPLAYFORM3 3. This equality follows since for each i ∈ {1, 3}, l ∈ {2, 4} and j ∈ W DISPLAYFORM4 (since x l will remain the maximal direction). Therefore, DISPLAYFORM5 In the second case, where we have DISPLAYFORM6 t · x i < η for i ∈ {1, 3}. Note that by Lemma 8.6, any DISPLAYFORM7. By all these observations, we have DISPLAYFORM8 By Eq. 14 and Eq. 15, it follows that, Z DISPLAYFORM9. Applying these observations b times, we see that Y DISPLAYFORM10 4) where the equality follows by Lemma 8.4. By Lemma 8.9, we have S DISPLAYFORM11 Hence, we can conclude that DISPLAYFORM12 Proof. Define DISPLAYFORM13 First note that by Lemma 8.4 we have W DISPLAYFORM14 where the second equality follows by Lemma 8.4. DISPLAYFORM15 To see this, note that by Lemma 8.6 and Lemma 8.5 it holds that u DISPLAYFORM16 0 · x 2 and thus Eq. 16 holds. Now assume that j ∈ U + T (l) for l ∈ {2, 4}. Then DISPLAYFORM17 Proof. The claim holds for t = 0. Consider an iteration T. If DISPLAYFORM18 η, where the last inequality follows from the previous observation. Hence, DISPLAYFORM19 The proof of the second claim follows similarly. DISPLAYFORM20 The third claim holds by the following identities and bounds DISPLAYFORM21 η by the previous claims. We are now ready to prove a global optimality guarantee for gradient descent. Proposition 8.13. Let k > 16 and γ ≥ 1. With probabaility at least 1 − DISPLAYFORM0 iterations, gradient descent converges to a global minimum. Proof. First note that with probability at least 1 − DISPLAYFORM1 −8 the claims of Lemma 8.1 and Lemma 8.2 hold. Now, if gradient descent has not reached a global minimum at iteration t then either DISPLAYFORM2 where the last inequality follows by Lemma 8.1. DISPLAYFORM3 by Lemma 8.9. However, by Lemma 8.10, it follows that after 5 consecutive iterations t < t < t + 6 in which DISPLAYFORM4 To see this, first note that for all t, N Wt (x +) ≤ γ+3c η by Lemma 8.12. Then, by Lemma 8.10 we have DISPLAYFORM5 where the second inequality follows by Lemma 8.1 and the last inequality by the assumption on k. Assume by contradiction that GD has not converged to a global minimum after T = 7(γ+1+8cη) DISPLAYFORM6 iterations. Then, by the above observations, and the fact that S + 0 > 0 with probability 1, we have DISPLAYFORM7 However, this contradicts Lemma 8.12. We will first need the following three lemmas. Lemma 8.14. With probability at least 1 − 4e −8, it holds that DISPLAYFORM0 Proof. The proof is similar to the proof of Lemma 8.1.Lemma 8.15. Assume that gradient descent converged to a global minimum at iteration T. Then there exists an iteration T 2 < T for which S DISPLAYFORM1 Proof. Assume that for all 0 ≤ t ≤ T 1 it holds that N Wt (x +) < γ and −N Wt (x −) < 1. By continuing the calculation of Lemma 8.3 we have the following: DISPLAYFORM2 2. For i ∈ {2, 4} and j ∈ W + 0 (i), it holds that w DISPLAYFORM3 Therefore, there exists an iteration DISPLAYFORM4 It suffices to show that for all T 1 ≤ t < T 2 the following holds: DISPLAYFORM5 The first claim follows since at any iteration N Wt (x +) can decrease by at most 2ηk = 2c η. For the second claim, let t < t be the latest iteration such that N W t (x +) ≥ γ. Then at iteration t it holds that −N W t (x −) < 1 and N W t (x +) ≥ γ. Therefore, for all i ∈ {1, 3}, l ∈ {2, 4} and DISPLAYFORM6 t + ηx l. Hence, by Lemma 8.5 and Lemma 8.6 it holds that U + t +1 ∪ U + t +1 = ∅. Therefore, by the gradient update in Eq. 11, for all 1 ≤ j ≤ k, and all t < t ≤ t we have u DISPLAYFORM7 The above argument shows that DISPLAYFORM8 Assume that k ≥ 64 and gradient descent converged to a global minimum at iteration T. Then, DISPLAYFORM9 Proof. Notice that by the gradient update in Eq. 10 and Lemma 8.2, X + t can be strictly larger than max DISPLAYFORM10. We know by Lemma 8.15 that there exists T 2 < T such that S + T2 ≥ γ+1−3c η and that N Wt (x +) < γ and −N Wt (x −) ≥ 1 only for t > T 2. Since S + t ≤ γ + 1 + 8c η for all t by Lemma 8.12, there can only be at most DISPLAYFORM11 where the second inequality follows by Lemma 8.1 and the third inequality by the assumption on k. We are now ready to prove the main of this section. Assume without loss of generality that z i = (−1, −1) = x 3. Define Notice that DISPLAYFORM0 Furthermore, by Lemma 8.7 we have DISPLAYFORM1 and by Lemma 8.14, DISPLAYFORM2. Combining this fact with Eq. 19 and Eq. 20 we get DISPLAYFORM3 which implies together with Eq. 18 that X DISPLAYFORM4 where the first inequality is true because DISPLAYFORM5 The second inequality in Eq. 21 follows since P + T ≤ c η and by appyling Lemma 8.16. Finally, the last inequality in Eq. 21 follows by the assumption on k.10 Hence, z is classified correctly. We will need the following lemmas. Lemma 8.18. With probability at least 1 − 8e −8, it holds that DISPLAYFORM0 Under review as a conference paper at ICLR 2019Proof. The proof is similar to the proof of Lemma 8.1 and follows from the fact that DISPLAYFORM1 Lemma 8.19. Let DISPLAYFORM2 Then for all t, there exists X, Y ≥ 0 such that |X| ≤ η U + 0, |Y | ≤ η U + 0 and DISPLAYFORM3 Proof. First, we will prove that for all t there exists a t ∈ Z such that for DISPLAYFORM4 11 We will prove this by induction on t. For t = 0 this clearly holds. Assume it holds for an iteration t. Let j 1 ∈ U − 0 and j 2 ∈ U − 0. By the induction hypothesis, there exists a T ∈ Z such that u DISPLAYFORM5. In either case, by Eq. 11, we have the following update at iteration t + 1: DISPLAYFORM6 where a ∈ {−1, 0, 1}. Hence, u DISPLAYFORM7 − (a t + a)ηx 2. This concludes the proof by induction. Now, consider an iteration t, j 1 ∈ U + 0, j 2 ∈ U + 0 and the integer a t defined above. If a t ≥ 0 then DISPLAYFORM8 ) which proves the claim in the case that a t ≥ 0.If a t < 0 it holds that 11 Recall that by Lemma 8.5 we know that DISPLAYFORM9 Under review as a conference paper at ICLR 2019 DISPLAYFORM10 Since for all 1 ≤ j ≤ k it holds that u DISPLAYFORM11 4) which concludes the proof. Lemma 8.20. Let DISPLAYFORM12 Then for all t, DISPLAYFORM13 Proof. We will first prove that for all t there exists an integer a t ≥ 0 such that for DISPLAYFORM14 · x 4 + ηa t. We will prove this by induction on t. For t = 0 this clearly holds. Assume it holds for an iteration t. DISPLAYFORM15. By the induction hypothesis, there exists an integer a t ≥ 0 such that u DISPLAYFORM16, it follows that if a t ≥ 1 we have the following update at iteration T + 1: DISPLAYFORM17 where a ∈ {−1, 0, 1}. Hence, u DISPLAYFORM18 0 ·x 2 +η(a t +a) and u DISPLAYFORM19. This concludes the proof by induction. Now, consider an iteration t, DISPLAYFORM20 and the integer a t defined above. We have, DISPLAYFORM21 It follows that DISPLAYFORM22 which concludes the proof. We are now ready to prove the main of this section. Proof. With probability at least 1 − √ 2k √ πe 8k − 16e −8 Proposition 8.13 and Lemma 8.18 hold. It suffices to show generalization on negative points. Assume that gradient descent converged to a global minimum at iteration T. Let (z, −1) be a negative point. Assume without loss of generality that z i = x 2 for all 1 ≤ i ≤ d. Define the following sums for l ∈ {2, 4}, DISPLAYFORM23 First, we notice that DISPLAYFORM24 We note that by the analysis in Lemma 8.18, it holds that for any t, j 1 ∈ U + 0 and j 2 ∈ U + 0, either j 1 ∈ U + t and j 2 ∈ U + t, or j 1 ∈ U + t and j 2 ∈ U + t. We assume without loss of generality that j 1 ∈ U + T and j 2 ∈ U + T. It follows that in this case DISPLAYFORM25 12 Otherwise we would replace Y − T with Y − T and vice versa and continue with the same proof. DISPLAYFORM26. By Lemma 8.20 and Lemma 8.18 DISPLAYFORM27 and by Lemma 8.19 and Lemma 8.18 there exists Y ≤ c η such that: DISPLAYFORM28 Plugging these inequalities in Eq. 24 we get: DISPLAYFORM29 By Lemma 8.16 we have X − T ≤ 34c η. Hence, by using the inequality S − T ≤ c η we conclude that DISPLAYFORM30 where the last inequality holds for k > 64 We will now prove pattern detection . In the case of over-paramterized networks, in Proposition 8.17 we proved that DISPLAYFORM31 1+α(k). Since for i ∈ {1, 3} it holds that D xi ≥ X + (i), it follows that patterns x 1 and x 3 are detected. Similarly, in Proposition 8.21our analysis implies that, without loss of generality, DISPLAYFORM32 (under the assumption that we assumed without loss of generality), it follows that patterns x 2 and x 4 are detected. The confidence of the detection is at 2 by a symmetry argument. This will finish the proof of the theorem. For the proof, it will be more convenient to denote the matrix of weights at iteration t as a tuple of 4 vectors, i.e., W t = w To see this, we will illustrate this through one case, the other cases are similar. Assume, for example, that arg max 1≤l≤4 u(FORMULA0 t · x l = 3 and arg max l∈{2,4} u t · x l = 2 and assume without loss of generality that N W T · x 1 ≤ 2c η and therefore, x 1 cannot be detected with confidence greater than 2c η. 2. Let Z 1 be the set of positive points which contain only the patterns x 1, x 2, x 4, Z 2 be the set of positive points which contain only the patterns x 3, x 2, x 4. Let Z 3 be the set which contains the negative point with all patterns equal to x 2 and Z 4 be the set which contains the negative point with all patterns equal to x 4. By the proof of the previous section, if the event E holds, then there exists 1 ≤ i ≤ 4, such that gradient descent converges to a solution at iteration T which errs on all of the points in Z i. Therefore, its test error will be at least p * (recall Eq. 3). In this section we assume that we are given a training set S ⊆ {±1} 2 × {±1} 2 consisting of points (x 1, 1), (x 2, −1), (x 3, 1), (x 4, −1), where x 1 =, x 2 = (−1, 1), x 3 = (−1, −1) and x 4 = (1, −1). Our goal is to learn the XOR function with gradient descent. Note that in the case of two dimensions, the convolutional network introduced in Section 3 reduces to the following two-layer fully connected network. DISPLAYFORM0 We consider running gradient descent with a constant learning rate η ≤ cη k, c η ≤ 1 and IID gaussian initialization with mean 0 and standard deviation σ g = cη 16k 3/2. We assume that gradient descent minimizes the hinge loss (W) = where optimization is only over the first layer. We will show that gradient descent converges to the global minimum in a constant number of iterations. For each point x i ∈ S define the following sets of neurons: DISPLAYFORM1 t · x i < 0

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We show in a simplified learning task that over-parameterization improves generalization of a convnet that is trained with gradient descent.

We introduce a type of neural network that is structurally resistant to adversarial attacks, even when trained on unaugmented training sets.  The resistance is due to the stability of network units wrt input perturbations.


Question: Represent the scientific article to find a short TLDR

Question: We introduce a new and rigorously-formulated PAC-Bayes few-shot meta-learning algorithm that implicitly learns a model prior distribution of interest. Our proposed method extends the PAC-Bayes framework from a single task setting to the few-shot meta-learning setting to upper-bound generalisation errors on unseen tasks. We also propose a generative-based approach to model the shared prior and task-specific posterior more expressively compared to the usual diagonal Gaussian assumption. We show that the models trained with our proposed meta-learning algorithm are well calibrated and accurate, with state-of-the-art calibration and classification on mini-ImageNet benchmark, and competitive in a multi-modal task-distribution regression. One unique ability of humans is to be able to quickly learn new tasks with only a few training examples. This is due to the fact that humans tend to exploit prior experience to facilitate the learning of new tasks. Such exploitation is markedly different from conventional machine learning approaches, where no prior knowledge (e.g. training from scratch with random initialisation) , or weak prior knowledge (e.g., fine tuning from pre-trained models) are used when encountering an unseen task for training. This motivates the development of novel learning algorithms that can effectively encode the knowledge learnt from training tasks, and exploit that knowledge to quickly adapt to future tasks . Prior knowledge can be helpful for future learning only if all tasks are assumed to be distributed according to a latent task distribution. Learning this latent distribution is, therefore, useful for solving an unseen task, even if the task contains a limited number of training samples. Many approaches have been proposed and developed to achieve this goal, namely: multi-task learning , domain adaptation and meta-learning . Among these, meta-learning has flourished as one of the most effective methods due to its ability to leverage the knowledge learnt from many training tasks to quickly adapt to unseen tasks. Recent advances in meta-learning have produced state-of-the-art in many benchmarks of few-shot learning data sets (; ; ; ; ;). Learning from a few examples is often difficult and easily leads to over-fitting, especially when no model uncertainty is taken into account. This issue has been addressed by several recent Bayesian meta-learning approaches that incorporate model uncertainty into prediction, notably LLAMA that is based on Laplace method , or PLATIPUS , Amortised Meta-learner and VERSA that use variational inference (VI). However, these works have not thoroughly investigated the generalisation errors for unseen samples, ing in limited theoretical generalisation guarantees. Moreover, most of these papers are based on variational functions that may not represent well the richness of the underlying distributions. For instance, a common choice for the variational function relies on the diagonal Gaussian distribution, which can potentially worsen the prediction accuracy given its limited representability. In this paper, we address the two problems listed above with the following technical novelties: (i) derivation of a rigorous upper-bound for the generalisation errors of few-shot meta-learning using PAC-Bayes framework, and (ii) proposal of a novel variational Bayesian learning based on implicit The few-shot meta-learning problem is modelled using a hierarchical model that learns a prior p(w i ; θ) using a few data points s ij )}. Shaded nodes denote observed variables, while white nodes denote hidden variables. generative models to facilitate the learning of unseen tasks. Our evaluation shows that the models trained with our proposed meta-learning algorithm is at the same time well calibrated and accurate, with competitive in terms of Expected Calibration Error (ECE) and Maximum Calibration Error (MCE), while outperforming state-of-the-art methods in a few-shot classification benchmark (mini-ImageNet). Our paper is related to Bayesian few-shot meta-learning techniques that have been developed to incorporate uncertainty into model estimation. LLAMA employs the Laplace method to extend the deterministic estimation assumed in MAML to a Gaussian distribution. However, the need to estimate and invert the Hessian matrix makes this approach computationally challenging for large-scale models, such as deep neural networks. Variational inference (VI) addresses such scalability issue -remarkable examples of VI-based methods are PLATIPUS, BMAML , Amortised meta-learner and VERSA . Although these VI-based approaches have demonstrated impressive in regression, classification as well as reinforcement learning, they do not provide any theoretical guarantee on generalisation errors for unseen samples within a task. Moreover, the overly-simplified family of diagonal Gaussian distributions used in most of these works limits the expressiveness of the variational approximation, ing in a less accurate prediction. Our work is also related to the PAC-Bayes framework used in multi-task learning that provides generalisation error bounds with certain confidence levels. These previously published papers jointly learn a single shared prior and many task-specific posteriors without relating the shared prior to any task-specific posterior. Hence, these approaches need to store all task-specific posteriors, ing in un-scalable solutions, especially when the number of tasks is large. In contrast, our proposed method learns only the shared prior of model parameters and uses that prior to estimate the task-specific posterior through the likelihood function by performing a fixed number of gradient updates. This proposed variant of amortised inference allows a memory efficient solution, and therefore, more favourable for applications with large number of tasks, such as few-shot meta-learning. In this section, we first define and formulate the few-shot meta-learning problem. Subsequently, we derive the generalisation upper-bound based on PAC-Bayes framework. We then present our proposed approach that employs implicit variational distributions for few-shot meta-learning. We use the notation of task environment to describe the unknown distribution p(T) over a family of tasks, from where tasks are sampled. Each task T i in this family is indexed by i ∈ {1, ..., T} and associated with a dataset {X i, Y i} consisting of a training/support set {X ij given the small support set for task T i . We rely on a Bayesian hierarchical model as shown in Figure 1, where w i represents the model parameters for task T i, and θ denotes the meta-parameters shared across all tasks. For example, in MAML , w i are the neural network weights for task T i that is initialised from θ and obtained by performing truncated gradient descent using {X While the conventional graphical model methods in meta-learning learn the joint probability p(θ, w 1:T |Y 1:T, X 1:T) (, Section A. 3), our objective function for the few-shot meta-learning is to minimise the negative log predictive probability w.r.t. the meta-parameters θ as follows: where we simplify the notation by dropping the explicit dependence on X from the set of conditioning variables (this simplification is adopted throughout the paper). The predictive probability term inside the expectation in can be expanded by applying the sum rule of probability and lower-bounded by Jensen's inequality: In practice, the task-specific posterior p(w i |Y (t) i; θ) is often intractable, and therefore, approximated by a distribution q(i, θ). Given this assumption and the in, the upper bound of the objective function in can be presented as: where: Hence, instead of minimising the objective function in, we minimise the upper bound in. There are two issues related to the optimisation of this upper bound: (i) the generalisation error for (x i}, and (ii) how to estimate q(w i ; λ i) that can approximate the true posterior p(w i |Y (t) i; θ) accurately, so that we can evaluate and minimise the upper-bound in. We address the generalisation error in Section 3.2 and present a variational method to obtain an expressive variational posterior q(w i ; λ i) in Section 3.3. We first introduce the PAC-Bayes bound for the single-task problem in Theorem 1. Theorem 1 (PAC-Bayes bound for single-task setting ). Let D be an arbitrary distribution over an example domain Z. Let H be a hypothesis class,: H × Z → be a loss function, π be a prior distribution over H, and δ ∈. is an i.i.d. training set sampled according to D, then for any "posterior" Q over H, the following holds: where Theorem 1 indicates that with a high probability, the expected error of an arbitrary posterior Q on data distribution p(z) is upper-bounded by the empirical error plus a complexity regularisation term. These two terms express the trade-off between fitting data (bias) and regularising model complexity (variance). Remark 1. Despite the assumption based on bounded loss function, the PAC-Bayes bound can also be extended to unbounded loss function (, Section 5). Before presenting the novel bound for few-shot meta-learning, we define some notations. Recall that m is the number of samples in the query set {X The novel bound on the generalisation error for the few-shot meta-learning problem is shown in Theorem 2. Please refer to Appendix A for the proof. Theorem 2 (PAC-Bayes bound for few-shot meta-learning in). For the general error of few-shot meta-learning in, the following holds: Remark 2. The derived in Theorem 2 is different from the one in (, Theorem 2). As mentioned in Section 2, the prior work does not relate the posterior of model parameters q(w i ; λ i) to the shared prior p(w i ; θ). The "hypothesis" in that case is a tuple including the model parameters sampled from the prior and task-specific posterior. In contrast, our approach is a variant of amortised inference that relates the posterior from the prior and likelihood function by gradient updates (see Section 3.3). Hence, the "hypothesis" in our case includes the parameters sampled from the task-specific posterior only. The discrepancy of the "hypothesis" used between the two approaches in different upper-bounds, particularly at the regularisation term. Given the in Theorem 2, the objective function of interest is to minimise the generalisation upper-bound: As denoted in Section 3.1, q(w i ; λ i) is a variational posterior that approximates the true posterior p(w i |Y (t) i; θ) for task T i, and therefore, can be obtained by minimising the following KL divergence: The ing cost function (excluding the constant term) in is often known as the variational free energy (VFE). For simplicity, we denote the cost function as The first term of VFE can be considered as a regularisation that penalises the difference between the shared prior p(w i ; θ) and the variational task-specific posterior q(w i ; λ i), while the second term is referred as data-dependent or likelihood cost. Exactly minimising the cost function in is computationally challenging, so gradient descent is used with θ as the initialisation: where α t is the learning rate and the truncated gradient descent consists of a single step (the extension to a larger number of steps is trivial). Given the approximated posterior q(w i ; λ i) with parameter λ i obtained from, we can calculate and optimise the generalisation upper bound in w.r.t. θ. In Bayesian statistics, the shared prior p(w i ; θ) represents a modelling assumption, and the variational task-specific posterior q(w i ; λ i) is a flexible function that can be adjusted to achieve a good trade-off between performance and complexity. In general, p(w i ; θ) and q(w i ; λ i) can be modelled using two general types of probabilistic models: prescribed and implicit . For example, Amortised Meta-learner is a prescribed approach where both distributions are assumed to be diagonal Gaussians. In this paper, we present a more expressive way of implicitly modelling the shared prior and task-specific posterior. Both distributions p(w i ; θ) and q(w i ; λ i) are now defined at a more fundamental level whereby data is generated through a stochastic mechanism without specifying parametric distributions. We use a parameterised model (i.e., a generator G represented by a deep neural network) to model the sample generation from the prior and posterior: where p(z) is usually denoted by a Gaussian model N (0, I) or a uniform model U. Due to the nature of implicit models, the KL divergence term in, in particular the density ratio q(w i ; λ i) /p(w i ; θ), cannot be evaluated either analytically or symbolically. We, therefore, propose to employ the probabilistic classification approach (, Chapter 4) to estimate the KL divergence term. We use a parameterised model -a discriminator D represented by a deep neural network -as a classifier to distinguish different w i sampled from the prior p(w i ; θ) (label 1) or the posterior q(w i ; λ i) (label 0). The objective function to train the discriminator D can be written as: where ω i is the parameters of D for task T i. Given the discriminator D, the KL divergence term in can be estimated as: where z (l) ∼ p(z), L t is the number of Monte Carlo samples, and V (., ω i) is the output of the discriminator D without sigmoid activation. The variational-free energy in can, therefore, be rewritten as: One problem that arises when estimating the loss in is how to obtain the local optimal parameters ω * i for the discriminator D. One simple approach is to generate several model parameters w i from the prior p(w i ; θ) and posterior q(w i ; λ i) following to train D(.; ω i) by optimising the cost in. The downside is the significant increase in training time and memory usage to store the computational graph to later be used for minimising the upper-bound in w.r.t. θ. To overcome this limitation, we propose to meta-learn ω i using MAML . In this scenario, we define ω 0 as the meta-parameters (or initialisation) of ω i. Within each task, we initialise ω i at ω 0 and use the generated w i from as training data. This approach leads to our proposed algorithm, named Statistical Implicit Bayesian Meta-Learning (SImBa), shown in Algorithm 1. Our assumption here is that the discriminator can provide an optimal estimate of the KL divergence term as shown in. This strong theoretical property only holds when the discriminator model is correctly-specified (, Remark 4.7). To this end, we employ the universal approximation theorem to model the discriminator as a feed-forward fully connected neural network. We expect that under this modelling approach, the discriminator model is approximately correctly-specified. Output: meta-parameters θ of the shared prior p(w i ; θ), and discriminator meta-parameters ω 0 1: initialise θ and ω 0 2: while θ not converged do repeat step 7 to calculate discriminator loss L D (ω i) end for 15: Another approach to estimate the KL divergence term in is to use a lower bound of fdivergence . There is a difference between the lower bound approach and the probabilistic classification presented in this subsection. In the former approach, the lower bound of the KL divergence is maximised to tighten the bound. In the latter approach, a discriminator is trained to minimise the logistic regression loss to estimate the ratio q(w i ; λ i) /p(w i ; θ), and use Monte Carlo sampling to approximate the KL divergence of interest. One potential drawback of the implicit modelling used in this paper is the curse of dimensionality, ing in an expensive computation during training. This is an active research question when dealing with generative models in general. This issue can be addressed by encoding the highdimensional data, such as images, to a feature embedding space by supervised-learning on the same training data set . This strategy reduces the dimension of the input space, leading to smaller generator and discriminator models. The trade-off lies in the possibility of losing relevant information that can affect the performance on held-out tasks. It is also worthy noting that our proposed method is easier to train than prior Bayesian few-shot meta-learning ) because we no longer need to estimate the weighting factor of the KL divergence term in. The trade-off of our approach lies in the need to set the significance level δ, but tuning δ is arguably more intuitive than estimating the correct weighting factor for the KL divergence term. We evaluate SImBa in both few-shot regression and classification problems. We also compare to prior state-of-art meta-learning methods to show the strengths and weaknesses of SImBa. The experiment in this subsection is a multi-modal task distribution where half of the data is generated from sinusoidal functions, while the other half is from linear functions. The details of the experimental setup and additional visualisation are presented in Appendix B. The in Figure 2 (leftmost and middle graphs) show that SImBa is able to vary the prediction variance, especially when there is more uncertainty in the training data, while MAML can only output a single value at each data point. To further evaluate the predictive uncertainty, we employ the reliability diagram based on the quantile calibration for regression . The reliability diagram shows a correlation between predicted and actual probability. A perfect calibrated model will have its predicted probability equal to the actual probability, and hence, align well with the diagonal y = x. Figure 2 (Right) shows the for SImBa and some published meta-learning methods. As expected, Bayesian metalearning approaches, and in particular, Amortised Meta-learner, which relies on diagonal Gaussian distributions, are better calibrated than MAML -a deterministic approach. However, the averaged slope of the Amortised Meta-learner correlation curve is quite small, implying that its predicted probability is peaked at the mean of the ground-truth distribution with small covariances. In contrast, SImBa employs a much richer variational distribution, and therefore, ing in a model with better calibration. We evaluate SImBa on the N -way k-shot setting, where a meta learner is trained on many related tasks containing N classes with k examples per class. We use the train-test split that consists of 64 classes for training, 16 for validation, and 20 for testing . Please refer to Appendix C for the details of the model used. Although we target the estimation of model uncertainty, we also present the accuracy of SImBa against the state of the art on mini-ImageNet . The in Table 1 shows that SImBa achieves state-of-the-art in 1-shot setting when the base model is the 4-layer convolutional neural network (CNN) , and in 5-shot setting when different network architecture is used. We also show in Appendix D that generators with larger networks tend to classify better. Similar to the experiment for regression, we use reliability diagrams to evaluate the predictive uncertainty. The reliability diagrams show how well calibrated a model is when testing across many unseen tasks. A perfectly calibrated model will have its values overlapped with the identity function y = x, indicating that the probability associated with the label prediction is the same as the true probability. Figures 3a and 3b show the of SImBa and other Bayesian meta-learning methods. Visually, the model trained with SImBa shows better calibration than the ones trained with MAML and PLATIPUS, while being competitive to Amortised Meta-learner. To further evaluate, we compute the expected calibration error (ECE) and maximum calibration error (MCE) of the models trained with these methods. The plotted in Figure 3c show that the model trained with SImBa has smaller ECE and MCE compared to MAML and PLATIPUS. SImBa also has lower ECE and competitive MCE compared to Amortised Metalearner, but notice that Amortised Meta-learner has a worse classification than SImBa, as shown in Table 1. Table 1: The few-shot 5-way classification accuracy (in percentage, with 95% confidence interval) of SImBa averaged over 600 mini-ImageNet tasks are competitive to the state-of-the-art methods. SImBa outperforms other prior methods in 1-shot setting when using the standard 4-layer CNN, and 5-shot setting when using non-standard network architectures. Matching nets 43.56 ± 0.84 55.31 ± 0.73 Meta-learner LSTM 43.44 ± 0.77 60.60 ± 0.71 MAML 48.70 ± 1.84 63.15 ± 0.91 Prototypical nets 1 49.42 ± 0.78 68.20 ± 0.66 LLAMA 49.40 ± 1.83 PLATIPUS 50.13 ± 1.86 Amortised ML 45.00 ± 0.60 SImBa 51.01 ± 0.31 63.94 ± 0.43 Relation nets 50.44 ± 0.82 65.32 ± 0.70 VERSA 53.40 ± 1.82 67.37 ± 0.86 SNAIL 55.71 ± 0.99 68.88 ± 0.92 adaResNet 56.88 ± 0.62 71.94 ± 0.57 TADAM 58.50 ± 0.30 76.70 ± 0.30 LEO 61.76 ± 0.08 77.59 ± 0.12 LGM-Net 69. We introduce and formulate a new Bayesian algorithm for few-shot meta-learning. The proposed algorithm, SImBa, is based on PAC-Bayes framework which theoretically guarantees prediction generalisation on unseen tasks. In addition, the proposed method employs a generative approach that implicitly models the shared prior p(w i ; θ) and task-specific posterior q(w i ; λ i), ing in more expressive variational approximation compared to the usual diagonal Gaussian methods, such as PLATIPUS or Amortised Meta-learner . The uncertainty, in the form of the learnt implicit distributions, can introduce more variability into the decision made by the model, ing in well-calibrated and highly-accurate prediction. The algorithm can be combined with different base models that are trainable with gradient-based optimisation, and is applicable in regression and classification. We demonstrate that the algorithm can make reasonable predictions about unseen data in a multi-modal 5-shot learning regression problem, and achieve state-of-the-art calibration and classification with on few-shot 5-way tasks on mini-ImageNet data set. First, we present the two auxiliary lemmas that helps to prove Theorem 2. Lemma 1. For i = 1: n, if X i and Y i are random variables, then: Proof. The proof is quite direct: Hence, the proof. Lemma 2. For n events A i with i = 1: n, the following holds: Proof. Proof can be done by induction. For n = 2: Suppose that it is true for case n: We prove that this is also true for case (n + 1): It is, therefore, true for (n + 1), and hence, the proof. Secondly, we apply the PAC-Bayes bound in Theorem 1 on the task i to obtain an upper-bound for a single task i shown in Corollary 1. Corollary 1. For a single task T i in Eq. and δ i ∈, the following holds: Finally, we can employ Lemmas 1 and 2 combined with Corollary 1 to derive the novel upper-bound for few-shot meta-learning setting. Theorem 2 (PAC-Bayes bound for few-shot meta-learning in). For the general error of few-shot meta-learning in, the following holds: Proof. Applying the inequality in Lemma 1 by replacing where are defined at Eqs., and, respectively. Applying Lemma 2 the right hand side term of Ineq. gives: Applying the transitive property for Ineqs., and Corollary 1, and setting δ i = δ/T prove the theorem. The experiment is carried out with half of the data being generated from sinusoidal functions, while the other half from linear functions. The amplitude and phase of the sinusoidal functions are uniformly sampled from [0.1, 5] and [0, π], respectively, while the slope and intercept of the lines are sampled from [-3, 3]. Data is uniformly generated from [-5, 5], and the corresponding label is added a zero-mean Gaussian noise with a standard deviation of 0.3. Each task consists of 5 data points used for training (|Y i | = 15). The base model used in the regression experiment is a three-hidden fully connected layer neural network. Each hidden layer has 100 hidden units (1 → 40 → 40 → 40 → 1), followed by tanh activation. No batch normalisation is used. The generator is a fully connected network with two hidden layers consisting of 256 and 1024 units, respectively (dim(z) → 256 → 1024 → dim(w i)). The discriminator is also fully connected (dim(w i) → 512 → dim(z) → 1). These networks are activated by ReLU, except the last layer of the discriminator is activated by sigmoid function. No batch normalisation is used across these two networks. The variational parameters λ i and ω i are estimated by performing five gradient updates with learning rate α t = 0.001 and γ t = 0.001. The meta-parameters θ and the meta-parameter of the discriminator ω 0 are obtained with Adam with fixed step size α v = 10 −4 and γ v = 10 −5. At the beginning of training, we clip the gradient when updating λ i with a value of 10, and then gradually increase the clipping value. After 50,000 tasks, we remove the gradient clipping and continue to train until convergence. i | = 15N ). The latent noise z is a 100-dimensional vector sampled from a uniform distribution U. Adam optimiser is employed to optimise both θ and ω 0. Please refer to Table 2 for other hyper-parameters used. This model corresponds to the top part of Table 1. All input images are down-sampled to 84-by-84 pixels before performing experiments to be consistent with prior few-shot meta-learning works. The base model is a 4-block CNN, where each block consists of 32 filters with a size of 3-by-3, followed by a batch normalisation and a ReLU activation function. The generator is a 2-hiddenlayer fully connected network (dim(z) → 256 → 1024 → dim(w i)), where each layer is activated by ReLU without batch normalisation. The discriminator is also a fully connected network (dim(w i) → 1024 → 256 → dim(z) → 1) with ReLU activation and without batch normalisation (the last activation function is a sigmoid). This corresponds to the bottom part of Table 1. Here, we employ the features extracted from (, Section 4.2.2) as the encoding of the input images. The training for the feature embedding consists of 3 steps. First, raw input images are down-sampled to 80-by-80 pixels. Second, a wide residual neural network WRN-28-10 is trained with data and labels from the 64 classes of the training set. Finally, the intermediate features of 640 dimensions at layer 21 are chosen as the embedding features used for our classification experiments. The base model used in this experiment is a fully connected network with 1 hidden layer that consists of 128 hidden units (640 → 128 → N) followed by ReLU activation and batch normalisation. The generator model is constructed as a 1-hidden layer fully connected network with 512 hidden units, followed by ReLU without batch normalisation. The discriminator is also a fully connected network (dim(w i) → 512 → dim(z) → 1) with ReLU without batch normalisation. To study the effect of network architecture on the classification performance presented in Table 1, we repeat the classification experiment with the same setup, but different base networks. We vary the number of hidden units in the base network from 16 to 128, and also increase the size the the hidden layer of the generator from 256 to 512. The in Table 3 show that the larger the base network and the generator are, the better the classification accuracy.

Represent the short summary for finding the corresponding scientific article (long)

Bayesian meta-learning using PAC-Bayes framework and implicit prior distributions

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We show that posterior collapse in linear VAEs is caused entirely by marginal log-likelihood (not ELBO). Experiments on deep VAEs suggest a similar phenomenon is at play.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: As the area of Explainable AI (XAI), and Explainable AI Planning (XAIP), matures, the ability for agents to generate and curate explanations will likewise grow. We propose a new challenge area in the form of rebellious and deceptive explanations. We discuss how these explanations might be generated and then briefly discuss evaluation criteria. Explanations as a research area in AI (XAI) has been around for several decades BID7 BID5 BID10 BID45 BID12 BID40 BID44 BID28. It has additionally gained momentum recently as evidenced by the increasing number of workshops and special tracks covering it in various conferences (e.g., VIS-xAI, FEAPAI4Fin, XAIP, XAI, OXAI, MAKE-eXAI, ICCBR-19 Focus area).While still growing in use, there have been some approaches to formalizing XAI. BID11 stated that anything calling itself XAI should address the following questions:• Why did the agent do that and not something else?• When does the agent succeed and when does it fail?• When can I trust the agent?However, less thought out is the idea of explanations that are deceptive or rebellious in nature. These forms of explanation can be an entirely new area of discussion and use for certain autonomous agents. The study of deception and rebellion are both rich fields, and many aspects of both that are studied in civilian and military capacities. For example, the area of deception detection works on finding ways to detect inconsistencies BID41 BID22 BID2. BID17 discuss a number of ways why deception is an important topic for autonomous agents. Studies of rebellion and resistance have investigated how, why, when it does, and doesn't, happen (Martí and Copyright c 2019, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. BID24 BID33. The use of both has also been studied BID34 BID1 BID21 BID18 BID30.The idea of pairing deception and rebellion with explanations may not be intuitive initially. However, in addition to being areas of rich study, deception and rebellion offer key conditions that are of interest to agent reasoning. Predominately, it requires multiple actors (i.e., An actor deceives another actor, or an actor rebels against a coordinator). Additionally, there needs to be some sort of conflict or misalignment between the actors. Either something needs to be in contention for an actor to rebel, or something needs to be in conflict for the actor to use deception. b Rebellion in agents has been a growing area of interest BID9 BID3 BID8. This area is focused on finding models in which agents can rebel from directives given in certain circumstances. This can include having more upto-date knowledge that would affect the plan, finding opportunities to exploit but may be off-mission, or solving problems or roadblocks before they become an issue even if it is off-mission. discuss three ways in which rebellion could manifest in agents. The expression of a rebellion can consist of either an explicit or implicit act. The focus is either inward or outward facing. Lastly, the interaction initiation can either be reactive or proactive. Deception in agents has been progressing over the last decade, with many discussions on formalizing deception. The majority of this formalism is on the topic of lying BID42 BID38 BID43. There has also been inroads for more encompassing deception as described by BID39 and BID37. Of interest here, BID37 defined Quantitative & Qualitative Maxims for Dishonesty as the following maxims:1. Lie, Bullshit (BS), or withhold information as little as possible to achieve your objective.2. Never lie if you can achieve your objective by BS.3. Never lie nor BS if you can achieve your objective by withholding Information.4. Never lie, BS, nor withhold information if you can achieve your objective with a half-truth. A particular topic that has received attention is deceptive, or dishonest, agents in negotiations BID31 BID47.With these concepts in mind, we will pursue research to answer the following:What kind of reasoning models are required to generate explanations of a deceptive or rebellious nature? Research into rebellion and deception have been studied for decades in other fields. We summarize some of these studies in this section, as they will form the foundation and context of our approach. Dissent assumes a challenge to some system of power or belief BID26. The author goes further to differentiate between dissent and rebellion. Martin claims that a basic distinction between dissent and rebellion is that dissenters still believe in the process that has been established whereas rebels do not. A number of real-world studies on dissent and whistleblowing in organizations showcase the importance of this topic and area BID15 BID19 BID26 BID32 BID25. Many of these discuss the outcomes and towards both the actor and the agent in these situations. BID25 discuss organizational responses to whistle-blowing and dissent. Manager reprisals may include building a damaging record towards them, threats against them, isolation, prosecution, or setting them up for failure at their job. In a number of instances, it seems that giving dissent, or becoming a whistle-blower, can quickly turn into an adversarial or antagonistic situation. Closely tied to dissent is the idea of cynicism. BID23 defines it as: a belief that there is a gap between desired and observed organizational identity; a negative affect toward the organization or organizational change (strategy); and tendencies to disparaging and/or critical behaviors toward the organization that are consistent with those beliefs and affect. They additionally give examples of cynicism as: pessimism, emotional/narrative expressions or outbursts, frustration, irony, accusations, neglect, negative coping behaviors and expressions, and aggression. Observed actions of resistance to change have been studied in a number of ways. Once such study BID46 noted that there was a tendency towards "informal or mundane" types of resistance compared to an open, direct, and explicit form of objection. When faced with managerial initiatives, the following expressions of resistance were noted: "careful carelessness", humor, cynicism and skepticism, nostalgic talk (i.e., the "Good ol' Days"), alternative articulations of self-hood, and simulation of productivity. While these expressions of resistance were present, workers were also camouflaging this dissent with a goodhumored appearance. It was noted that hiding dissent allowed for behind-the-scenes inaction to stifle any new directives and also avoided many conversations that workers deemed futile. Similar studies include BID14 BID27 BID35.Along with giving a number of examples of resistance, BID24 While overt forms of resistance usually did not work, the stories of that resistance remained and were used later to continue forms of resistance. There have been several studies on how deception in society works. A very good resource is (Whiten and Byrne 1988) which defines a number of deceptive actions observed among primates in the wild. They are categorized as either concealment, distraction, use of a tool, or use of another agent. Examples include hiding things from sight, looking at things to have others avoid looking at something else, and getting other agents to take the blame for something. In addition to primates, there have been deceptive studies for cephalopods BID4 ) and dolphins BID16.In studies of deception in warfare BID34, there have been noticeable benefits to using deception. This includes minimizing the amount of resources for a task or ensuring the opponents mis-allocate their resources. Also, most of the example deceptive acts required an extended duration of time to be executed and prove successful. This included preparations for and performing the deceptive actions. A good case study from (Dunin-Keplicz and Verbrugge 2011) describes an ecological disaster scenario. We present it now to provide context for explanations discussed in Section 4. In the ecological disaster scenario, two poisons have breached containment in a large area. Each poison is highly dangerous on its own. However, when combined they could be explosive. Robots on the ground have the ability to neutralize specific poisons in small areas. An unmanned air vehicle (UAV) can survey large areas to identify high concentrations of poison. A coordinator can issue commands to either the UAV or the ground robots. A robot can gauge the type and quantity of poisons in the small area around it, and can move across the ground. The UAV can scan larger areas to identify large concentrations of individual poisons, though it reports only the highest quantity. Additionally, the UAV cannot scan areas that are obscured by ceilings or coverings. The coordinator receives the information from the UAV and the Robots, but does not have a view of their own. Examples of rebellion here could be robots not following commands to enter areas that would explode or the UAV deciding to survey a different area to help robots on the ground compared to an area that the coordinator instructed the UAV to survey. Let us now consider explanations that are rebellious or deceptive within the context given in Section 3. Some of the recurring reasons for generating these explanations include "makes it simpler" or "avoids a larger conversation". These can limit conversations but can also cause misinterpretations. It also has the ability to avoid conversation bottlenecks, letting agents continue to perform other actions. During a disaster, it may come to pass that an agent with the ability to administer first-aid or medical attention, perhaps they are equipped with air masks or oxygen, encounters a victim who will only let them near if they do not inform any law enforcement of their position. In this instance, the agent would administer the first aid or medical treatment and, when asked by the Coordinator to explain their activities, would say they are performing a different activity or in a different location. A robot could be traversing an area trying to help people evacuate or administer aid and they have informed the Coordinator they are performing this task. However, suppose there is a person who wishes to remain anonymous or would refuse help. In this case, the robot could explain its actions but leave out details that would identify the person in later reports or debriefs. This would help save the person and increase that victim's trust in the robot to help them out of the area. A version of an explanation with a half-truth could be as follows: a medical agent has found and is administering aid to a victim, however the victim is too far gone. Keeping the victim calm with a half-truth or "white lie" explanation would be beneficial to the victim. An example of a protest-based explanation could come from the following contingency. An exploratory or search agent has encountered an area of the environment that it deems too hazardous to continue in. The Coordinator asks why it isn't moving forward anymore. The agent responds with,"I will not move forward until the hazardous area has been secured and is safe to pass through. " As discussed in Section 2.3, cynicism is a bit odd. It is usually used as a form of soft-resistance. The agent still performs the action or command, but may not do it optimally. An example could be that the Coordinator assigns a robot that has a full amount of neutralizing and medical equipment to survey an area. This might take a while for the robot to execute, so the Coordinator might ask why progress is slow, and the explanation could be " If I could fly this would go much quicker." Alternatively, asking to release all of its equipment so that it can be lighter to perform the survey is another example. For this instance, the agent is in some sort of situation in which it will not continue an objective given by the Coordinator. Perhaps the Coordinator has tasked an agent with neutralizing an area under a fallen concrete roof. However the agent has noticed a victim in the area being treated by another agent. In that instance the agent could respond to the Coordinator, "I will not neutralize that area, there is a victim in the vicinity. Please assign me a different objective." In order to generate possible deceptive explanations as suggested above an agent would require a few things in its models to properly generate a model. An agent would require an internal model of the domain so that it can reason about possible actions, tasks, and goals. It would also require an internal model of the external agent's domain model. This is required so that when generating the "deceptive" aspect, it can be validated against the presumed model of that agent. In addition to these models, a few conditions should be met as well. Notably, a discrepancy must be noticed between the external agent's model and the internal model in relation to the query asked. There needs to be a specific condition or contingency in which the truth would not maximize an overall objective benefit for the agent in question. Likewise, rebellious explanations require similar things such as internal models for both the domain and objectives along with noticing a discrepancy between the objectives, the domain, and the agent's internal model. Of great interest is the work in model reconciliation BID6. This is focused on maintaining (at least) two separate models, an internal one to the agent, and an external one for someone interacting with the agent. The idea is for the agent to reconcile any differences between these two versions to formulate an explanation. This approach is promising in regards to expanding it towards rebel or deceptive tasks in explanation. In the case of either deceptive or rebellious explanations, a discrepancy is required. This has been an active research focus. survey work on discrepancy detection. Useful to this thread of research, BID29 ) discusses it in the context of goal reasoning. In terms of reasoning models, BID36 ) discuss some interesting concepts in relation to Goal Networks. A life cycle for goals is also discussed. Combining these goal networks and the temporal nature of goal life cycles, a goal timeline develops. This timeline structure can represent the reasoning needed for some of the explanation models once discrepancies have been detected. Utilizing models of the world that are not in the explainer's original model is both challenging and novel to pursue for several reasons. It requires an agent to distinguish between different viewpoints of explanations. Introduces reasoning over viable explanations that can be generated. Requires a conversation concerning the ethics of deciding when an agent can decide to deceive. Finally, it opens up the area of XAI to new sets of scenarios -namely those that are deceptive or rebellious. To facilitate the development of deceptive or rebellious explanations, we will need a way to evaluate them. We propose a few areas that may be suitable for such testing. One such testing ground is the RoboCup Rescue. This is a popular disaster simulation BID20 that can be leveraged to simulate examples similar to those given in Section 3. Various games and game simulations may prove useful to test for these explanations. Some game options include Minecraft, One Night Ultimate Werewolf, Secret Hitler, Clue, and Diplomacy. Other relevant domains may include those that involve unmanned air and underwater vehicles. These vehicles require a large amount of autonomy and can be utilized in areas where discrepancies between an operator's situation awareness and the vehicle's belief state differ dramatically. Along with testing simulations, we can also look at measures of explanation effectiveness. Some of these measures can include clarity, timeliness, or correctness. Did the explanation answer the query? How easy was the explanation to understand? Was the time it took to respond seen as adequate? Is the user's attitude toward the agent lower or higher given this form of explanation?

Represent the TLDR for finding the corresponding scientific article

Position paper proposing rebellious and deceptive explanations for agents.

Represent the short summary for finding the corresponding scientific article (long)

Argumentation frameworks are used to represent causality of plans/models to be utilized for explanations.