Question: represent the article for finding a single-sentence tldr

Question: Training methods for deep networks are primarily variants on stochastic gradient descent. Techniques that use (approximate) second-order information are rarely used because of the computational cost and noise associated with those approaches in deep learning contexts. However, in this paper, we show how feedforward deep networks exhibit a low-rank derivative structure. This low-rank structure makes it possible to use second-order information without needing approximations and without incurring a significantly greater computational cost than gradient descent. To demonstrate this capability, we implement Cubic Regularization (CR) on a feedforward deep network with stochastic gradient descent and two of its variants. There, we use CR to calculate learning rates on a per-iteration basis while training on the MNIST and CIFAR-10 datasets. CR proved particularly successful in escaping plateau regions of the objective function. We also found that this approach requires less problem-specific information (e.g. an optimal initial learning rate) than other first-order methods in order to perform well. Gradient-based optimization methods use derivative information to determine intelligent search directions when minimizing a continuous objective function. The steepest descent method is the most basic of these optimization techniques, but it is known to converge very slowly in ill-conditioned systems. Even outside of these cases, it still only has a linear rate of convergence. Newton's method is a more sophisticated approach -one that uses second-order derivative information, which allows the optimizer to model the error surface more accurately and thus take more efficient update steps. When it converges, it does so quadratically, but Newton's method also has limitations of its own. Firstly, it does not scale well: it can be very expensive to calculate, store, and invert the objective function Hessian. Secondly, the method may fail if the Hessian is indefinite or singular. A variety of methods have been developed to try and appropriate the strengths of each approach while avoiding their weaknesses. The conjugate gradient method, for example, uses only firstorder information but uses the history of past steps taken to produce a better convergence rate than steepest descent. Quasi-Newton methods, on the other hand, approximate the Hessian (or its inverse) using first-order information and may enforce positive-definiteness on its approximation. Other approaches like trust region methods use second-order information without requiring convexity. For further information about gradient-based optimization, see BID15.Deep learning (DL) provides a set of problems that can be tackled with gradient-based optimization methods, but it has a number of unique features and challenges. Firstly, DL problems can be extremely large, and storing the Hessian, or even a full matrix approximation thereto, is not feasible for such problems. Secondly, DL problems are often highly nonconvex. Thirdly, training deep networks via mini-batch sampling in a stochastic optimization problem. Even if the necessary expectations can be calculated (in an unbiased way), the variance associated with the batch sample calculations produces noise, and this noise can make it more difficult to perform the optimization. Finally, deep networks consist of the composition of analytic functions whose forms are known. As such, we can calculate derivative information analytically via back-propagation (i.e. the chain rule). These special characteristics of DL have motivated researchers to develop training methods specifically designed to overcome the challenges with training a deep neural network. One such approach is layer-wise pretraining BID1, where pretraining a neural network layer-by-layer encourages the weights to initialize close to a optimal minimum. Transfer learning works by a similar mechanism, relying on knowledge gained through previous tasks to encourage nice training on a novel task. Outside of pretraining, a class of optimization algorithms have been specifically designed for training deep networks. The Adam, Adagrad, and Adamax set of algorithms provide examples of using history-dependent learning rate adjustment BID6. Similarly, Nesterov momentum provides a method for leveraging history dependence in stochastic gradient descent BID17. One could possibly argue that these methods implicitly leverage second order information via their history dependence, but the stochastic nature of mini-batching prevents this from becoming explicit. Some researchers have sought to use second-order information explicitly to improve the training process. Most of these methods have used an approximation to the Hessian. For example, the L-BFGS method can estimate the Hessian (or its inverse) in a way that is feasible with respect to memory requirements; however, the noise associated with the sampling techniques can either overwhelm the estimation or require special modifications to the L-BFGS method to prevent it from diverging BID3. There have been two primary ways to deal with this: subsampling BID3 BID13 and mini-batch reuse BID16 BID12. Subsampling involves updating the Hessian approximation every L iterations rather than every iteration, as would normally be done. Mini-batch reuse consists of using the same minibatch on subsequent iterations when calculating the difference in gradients between those two iterations. These approximate second-order methods typically have a computational cost that is higher than, though on the same order of, gradient descent, and that cost can be further reduced by using a smaller mini-batch for the Hessian approximation calculations than for the gradient calculation BID2. There is also the question of bias: it is possible to produce unbiased low-rank Hessian approximations BID11, but if the Hessian is indefinite, then quasi-Newton methods will prefer biased estimates -ones that are positive definite. Other work has foregone these kinds of Hessian approximations in favor of using finite differences BID10 ). In this paper, we prove, by construction, that the first and second derivatives of feedforward deep learning networks exhibit a low-rank, outer product structure. This structure allows us to use and manipulate second-order derivative information, without requiring approximation, in a computationally feasible way. As an application of this low-rank structure, we implement Cubic Regularization (CR) to exploit Hessian information in calculating learning rates while training a feedforward deep network. Finally, we show that calculating learning rates in this fashion can improve existing training methods' ability to exit plateau regions during the training process. Second-order derivatives are not widely used in DL, and where they are used, they are typically estimated. These derivatives can be calculated analytically, but this is not often done because of the scalability constraints described in Section 1.1. If we write out the first and second derivatives, though, we can see that they have a low-rank structure to them -an outer product structure, in fact. When a matrix has low rank (or less than full rank), it means that the information contained in that matrix (or the operations performed by that matrix) can be fully represented without needing to know every entry of that matrix. An outer product structure is a special case of this, where an mxn matrix A can be fully represented by two vectors A = uv T. We can then calculate, store, and use secondorder derivatives exactly in an efficient manner by only dealing with the components needed to represent the full Hessians rather than dealing with those Hessians themselves. Doing this involves some extra calculations, but the storage costs are comparable to those of gradient calculations. In this section, we will illustrate the low-rank structure for a feedforward network, of arbitrary depth and layer widths, consisting of ReLUs in the hidden layers and a softmax at the output layer. A feedforward network with arbitrary activation functions has somewhat more complicated derivative formulae, but those derivatives still exhibit a low-rank structure. That structure also does not depend on the form of the objective function or whether a softmax is used, and it is present for convolutional and recurrent layers as well. The complete derivations for these cases are given in Appendix B.In our calculations, we make extensive use of index notation with the summation convention BID5. In index notation, a scalar has no indices (v), a vector has one index (v as v i or v i), a matrix has two (V as V ij, V i j, or V ij), and so on. The summation convention holds that repeated indices in a given expression are summed over unless otherwise indicated. For example, DISPLAYFORM0 The pair of indices being summed over will often consist of a superscript and a subscript; this is a bookkeeping technique used in differential geometry, but in this context, the subscripting or superscripting of indices will not indicate covariance or contravariance. We have also adapted index notation slightly to suit the structure of deep networks better: indices placed in brackets (e.g. the k in v (k),j ) are not summed over, even if repeated, unless explicitly indicated by a summation sign. A tensor convention that we will use, however, is the Kronecker delta: δ ij, δ i j, or δ ij. The Kronecker delta is the identity matrix represented in index notation: it is 1 for i = j and 0 otherwise. The summation convention can sometimes be employed to simplify expressions containing Kronecker deltas. For example, δ DISPLAYFORM1 Let us consider a generic feedforward network with ReLU activation functions in n hidden layers, a softmax at the output layer, and categorical cross-entropy as the objective function (defined in more detail in Appendix B. The first derivatives, on a per-sample basis, for this deep network are DISPLAYFORM2 where f is the per-sample objective function, v (k),j is the vector output of layer k, u, see Appendix B. In calculating these expressions, we have deliberately left ∂f ∂p j unevaluated. This keeps the expression relatively simple, and programs like TensorFlow BID0 can easily calculate this for us. Leaving it in this form also preserves the generality of the expression -there is no low-rank structure contained in ∂f ∂p j, and the low-rank structure of the network as a whole is therefore shown to be independent of the objective function and whether or not a softmax is used. In fact, as long as Equation 13 holds, any sufficiently smooth function of p j may be used in place of a softmax without disrupting the low-rank structure. The one quantity that needs to be stored here is η (n,k),j i for k = 1, 2,..., n − 1; it will be needed in the second derivative calculations. Note, however, that this is roughly the same size as the gradient itself. We can now see the low-rank structure: DISPLAYFORM3 is the outer product (or tensor product) of the vectors ∂f ∂p i and v (n),j, and DISPLAYFORM4 (which ends up being a rank-1 tensor) and v (k−1),j. The index notation makes the outer product structure clear. It is important to note that this low-rank structure only exists for each sample -a weighted sum of low-rank matrices is not necessarily (and generally, will not be) low rank. In other words, even if the gradient of f is low rank, the gradient of the expectation, F = E [f], will not be, because the gradient of F is the weighted sum of the gradients of f. The second-order objective function derivatives are then DISPLAYFORM5 DISPLAYFORM6 Calculating all of these second derivatives requires the repeated use of DISPLAYFORM7 Evaluating that Hessian is straightforward given knowledge of the activation functions and objective used in the network, and storing it is also likely not an issue as long as the number of categories is small relative to the number of weights. For example, consider a small network with 10 categories and 1000 weights. In such a case, DISPLAYFORM8 ∂p 2 would only contain 100 entries -the gradient would be 10 times larger. We now find that we have to store η (n,k),i j values in order to calculate the derivatives. In DISPLAYFORM9 ∂w 2, we also end up needing η (r,k),i j for r = n. In a network with n hidden layers, we would then have DISPLAYFORM10 matrices to store. For n = 10, this would be 45, for n = 20, this would be 190, and so on. This aspect of the calculations does not seem to scale well, but in practice, it is relatively simple to work around. It is still necessary to store η DISPLAYFORM11, r < n, only actually shows up in one place, and thus it is possible to calculate each η (r,k),i j matrix, use it, and discard it without needing to store it for future calculations. The key thing to note about these second derivatives is that they retain a low-rank structure -they are now tensor products (or the sums of tensor products) of matrices and vectors. For example, DISPLAYFORM12 With these expressions, it would be relatively straightforward to extract the diagonal of the Hessian and store or manipulate it as a vector. The rank of the weighted sum of low rank components (as occurs with mini-batch sampling) is generally larger than the rank of the summed components, however. As such, manipulating the entire Hessian may not be as computationally feasible; this will depend on how large the mini-batch size is relative to the number of weights. The low rank properties that we highlight here for the Hessian exist on a per-sample basis, as they did for the gradient, and therefore, the computational savings provided by this approach will be most salient when calculating scalar or vector quantities on a sample-by-sample basis and then taking a weighted sum of the . In principle, we could calculate third derivatives, but the formulae would likely become unwieldy, and they may require memory usage significantly greater than that involved in storing gradient information. Second derivatives should suffice for now, but of course if a use arose for third derivatives, calculating them would be a real option. Thus far, we have not included bias terms. Including bias terms as trainable weights would increase the overall size of the gradient (by adding additional variables), but it would not change the overall low-rank structure. Using the calculations provided in Appendix B, it would not be difficult to produce the appropriate derivations. Cubic Regularization (CR) is a trust region method that uses a cubic model of the objective function: DISPLAYFORM0 at the j-th iteration, where H j is the objective function Hessian and s j = x − x j. The cubic term makes it possible to use information in the Hessian without requiring convexity, and the weight σ j on that cubic term can have its own update scheme (based on how well m (s j) approximates f BID7 ). Solving for an optimal s j value then involves finding the root of a univariate nonlinear equation BID14. CR is not commonly used in deep learning; we have seen only one example of CR applied to machine learning BID7 and no examples with deep learning. This is likely the case because of two computationally expensive operations: calculating the Hessian and solving for s j. We can overcome the first by using the lowrank properties described above. The second is more challenging, but we can bypass it by using CR to calculate a step length (i.e. the learning rate) for a given search direction rather than calculating the search direction itself. Our approach in this paper is to use CR as a metamethod -a technique that sits on top of existing training algorithms. The algorithm calculates a search direction, and then CR calculates a learning rate for that search direction. For a general iterative optimization process, this would look like x j+1 = x j + α j g j, where g j is the search direction (which need not be normalized), α j is the learning rate, and the subscript refers to the iteration. With the search direction fixed, m would then be a cubic function of α at each iteration. Solving ∂m ∂α = 0 as a quadratic equation in α then yields DISPLAYFORM0 If we assume that g T ∇f < 0 (i.e. g is a descent direction), then α is guaranteed to be real. Continuing under that assumption, of the two possible α values, we choose the one guaranteed to be positive. The sampling involved in mini-batch training means that there are a number of possible ways to get a final α j g j . One option would be to calculate E [α j g j]. This would involve calculating an α value with respect to the search direction produced by each sample point and then averaging the product αg over all of the sample points. Doing this should produce an unbiased estimate of α j g j, but in practice, we found that this approach ed in a great deal sampling noise and thus was not effective. The second approach would calculate DISPLAYFORM1 To do this, we would calculate an α value with respect to the search direction produced by each sample point, as in the first option, calculate an average α value, and multiply the overall search direction by that average. This approach, too, suffered from excessive noise. In the interest of reducing noise and increasing simplicity, we chose a third option: once the step direction had been determined, we considered that fixed, took the average of g T Hg and g T ∇f over all of the sample points to produce m (α) and then solved for a single α j value. This approach was the most effective of the three. To test CR computationally, we created deep feedforward networks using ReLU activations in the hidden layers, softmax in the output layer, and categorical cross-entropy as the error function; we then trained them on the MNIST and CIFAR-10 (BID8 data sets. This paper shows from networks with 12 hidden layers, each 128 nodes wide. For the purposes of this paper, we treat network training strictly as an optimization process, and thus we are not interested in network performance measures such as accuracy and validation error -the sole consideration is minimizing the error function presented to the network. As we consider that minimization progress, we will also focus on optimization iteration rather than wall clock time: the former indicates the behaviour of the algorithm itself, whereas the latter is strongly dependent upon implementation (which we do not want to address at this juncture). Overall computational cost per iteration matters, and we will discuss it, but it will not be our primary interest. Further implementation details are found in Appendix A. Figure 1 shows an example of CR (applied on top of SGD). In this case, using CR provided little to no benefit. The average learning rate with CR was around 0.05 (a moving average with a period of 100 is shown in green on the learning rate plot both here and in the rest of the paper), which was close to our initial choice of learning rate. This suggests that 0.02 was a good choice of learning rate. Another reason the were similar, though, is that the optimization process did not run into any plateaus. We would expect CR to provide the greatest benefit when the optimization gets stuck on a plateau -having information about the objective function curvature would enable the algorithm to increase the learning rate while on the plateau and then return it to a more typical value once it leaves the plateau. To test this, we deliberately initialized our weights so that they lay on a plateau: the objective function is very flat near the origin, and we found that setting the network weights to random values uniformly sampled between 0.1 and -0.1 was sufficient.(a) Error (SGD in blue, SGD with CR in red) (b) Learning rate (calculated rate in red, period-100 moving average in green)Figure 3: Cubic Regularization (CR) applied to Stochastic Gradient Descent (SGD) on the CIFAR-10 Dataset; initial learning rate = 0.01, σ = 1000 Figure 2 shows the of SGD with and without CR when stuck on a plateau. There, we see a hundred-fold increase in the learning rate while the optimization is on the plateau, but this rate drops rapidly as the optimization exits the plateau, and once it returns to a more normal descent, the learning rate also returns to an average of about 0.05 as before. The CR calculation enables the training process to recognize the flat space and take significantly larger steps as a . Applying CR to SGD when training on CIFAR-10 (Figure 3) produced similar to those seen on MNIST.We then considered if this behaviour would hold true on other training algorithms: we employed CR with Adagrad BID4 and Adadelta on MNIST. The were similar. CR did not provide a meaningful difference when the algorithms performed well, but when those algorithms were stuck on plateaus, CR increased the learning rate and caused the algorithms to exit the plateau more quickly than they otherwise would have (Figures 4 and 5). The relative magnitudes of those increases were smaller than for SGD, but Adagrad and Adadelta already incorporate some adaptive learning rate behaviour, and good choices for the initial learning rate varied significantly from algorithm to algorithm. We also used a larger value for σ to account for the increased variability due to those algorithms' adaptive nature. The with Adadelta showed some interesting learning rate changes: the learning rate calculated by CR dropped steadily as the algorithm exited the plateau, but it jumped again around iteration 1200 as it apparently found itself in a flat region of space. We see this CR approach as an addition to, not a replacement for, existing training methods. It could potentially replace existing methods, but it does not have to in order to be used. Because of the low-rank structure of the Hessian, we can use CR to supplement existing optimizers that do not explicitly leverage second order information. The CR technique used here is most useful when the optimization is stuck on a plateau prior to convergence: CR makes it possible to determine whether the optimization has converged (perhaps to a local minimum) or is simply bogged down in a flat region. It may eventually be possible to calculate a search direction as well as a step length, which would likely be a significant advancement, but this would be a completely separate algorithm. We found that applying CR to Adagrad and Adadelta provided the same kinds of improvements that applying CR to SGD did. However, using CR with Adam BID6 did not provide gains as it did with the other methods. Adam generally demonstrates a greater degree of adaptivity than Adagrad or Adadelta; in our experiments, we found that Adam was better than Adagrad or Adadelta in escaping the plateau region. We suspect that trying to overlay an additional calculated learning rate on top of the variable-specific learning rate produced by Adam may create interference in both sets of learning rate calculations. Analyzing each algorithm's update scheme in conjunction with the CR calculations could provide insight into the nature and extent of this interference, and provide ways to further improve both algorithms. In future work, though, it would not be difficult to adapt the CR approach to calculate layer-or variable-specific learning rates, and doing that could address this problem. Calculating a variable-specific learning rate would essentially involve rescaling each variable's step by the corresponding diagonal entry in the Hessian; calculating a layer-specific learning rate would involve rescaling the step of each variable in that layer by some measure of the block diagonal component of the Hessian corresponding to those variables. The calculations for variable-specific learning rates with CR are given in Appendix B.There are two aspects of the computational cost to consider in evaluating the use of CR. The first aspect is storage cost. In this regard, the second-order calculations are relatively inexpensive (comparable to storing gradient information). The second aspect is the number of operations, and the second-order calculations circumvent the storage issue by increasing the number of operations. The number of matrix multiplications involved in calculating the components of Equation 9, for example, scales quadratically with the number of layers (see the derivations in Appendix B). Although the number of matrix multiplications will not change with an increase in width, the cost of naïve matrix multiplication scales cubically with matrix size. That being said, these calculations are parallelizable and as such, the effect of the computation cost will be implementation-dependent. A significant distinction between CR and methods like SGD has to do with the degree of knowledge about the problem required prior to optimization. SGD requires an initial learning rate and (usually) a learning rate decay scheme; an optimal value for the former can be very problem-dependent and may be different for other algorithms when applied to the same problem. For CR, it is necessary to specify σ, but optimization performance is relatively insensitive to this -order of magnitude estimates seem to be sufficient -and varying σ has a stronger affect on the variability of the learning rate than it does on the magnitude (though it does affect both). If the space is very curved, the choice of σ matters little because the step size determination is dominated by the curvature, and if the space if flat, it bounds the step length. It is also possible to employ an adaptive approach for updating σ BID7 ), but we did not pursue that here. Essentially, using CR is roughly equivalent to using the optimal learning rate (for SGD). In this paper, we showed that feedforward networks exhibit a low-rank derivative structure. We demonstrate that this structure provides a way to represent the Hessian efficiently; we can exploit this structure to obtain higher-order derivative information at relatively low computational cost and without massive storage requirements. We then used second-order derivative information to implement CR in calculating a learning rate when supplied with a search direction. The CR method has a higher per-iteration cost than SGD, for example, but it is also highly parallelizable. When SGD converged well, CR showed comparable optimization performance (on a per-iteration basis), but the adaptive learning rate that CR provided proved to be capable of driving the optimization away from plateaus that SGD would stagnate on. The were similar with Adagrad and Adadelta, though not with Adam. CR also required less problem-specific knowledge (such as an optimal initial learning rate) to perform well. At this point, we see it as a valuable technique that can be incorporated into existing methods, but there is room for further work on exploiting the low-rank derivative structure to enable CR to calculate search directions as well as step sizes. Starting at a point far from the origin ed in extremely large derivative and curvature values (not to mention extremely large objective function values), and this could sometimes cause difficulties for the CR method. This was easy to solve by choosing an initialization point relatively near the origin; choosing an initialization relatively near the origin also provided a significantly better initial objective function value. We initialized the networks' weights to random values between an upper and lower bound: to induce plateau effects, we set, the bounds to ±0.1, otherwise, we set them to ±0.2.All of the networks used a mini-batch size of 32 and were implemented in TensorFlow BID0. The initial learning rate varied with network size; we chose learning rates that were large and reasonable but perhaps not optimal, and for optimization algorithms with other parameters governing the optimization, we used the default TensorFlow values for those parameters. For the learning rate decay, we used an exponential decay with a decay rate of 0.95 per 100 iterations. The σ value used is specified along with the initial learning rate for each network's . This value was also not optimized but was instead set to a reasonable power of 10.B LOW-RANK DERIVATIONS FOR DEEP NETWORKS B.1 FEEDFORWARD NETWORK WITH RELU ACTIVATIONS TAB1 provides a nomenclature for our deep network definition. Equations 10-16 define a generic feedforward network with ReLU activation functions in the hidden layers, n hidden layers, a softmax at the output layer, and categorical cross-entropy as the objective function. DISPLAYFORM0 A (z) = max (z, 0) DISPLAYFORM1 DISPLAYFORM2 The relevant first derivatives for this deep network are DISPLAYFORM3 where there is no summation over j in Equation FORMULA13. We now define several intermediate quantities to simplify the derivation process: DISPLAYFORM4 DISPLAYFORM5 where there is no summation over j in Equations 19 and 20. We can now complete our calculations of the first derivatives. DISPLAYFORM6 DISPLAYFORM7 We then start our second derivative calculations by considering some intermediate quantities: DISPLAYFORM8 Convolutional and recurrent layers preserve the low-rank derivative structure of the fully connected feedforward layers considered above, and we will show this in the following sections. Because we are only considering a single layer of each, we calculate the derivatives of the layer outputs with respect to the layer inputs -in a larger network, those derivatives will be necessary for calculating total derivatives via back-propagation. We can define a convolutional layer as DISPLAYFORM0 where x i j is the layer input, σ is the vertical stride, τ is the horizontal stride, A is the activation function, and v s t is the layer output. A convolutional structure can make the expressions somewhat complicated when expressed in index notation, but we can simplify matters by using the simplification z with no summation over s and t in any of the expressions above. Using the simplification with z sl tk makes it significantly easier to see the low rank structure in these derivatives, but that structure is still noticeable without the simplification. The conditional form of the expressions is more complicated, but it is also possible to see how the derivatives relate to w DISPLAYFORM1 where t indicates the number of times that the recursion has been looped through. If we inspect this carefully, we can actually see that this is almost identical to the hidden layers of the feedforward network: they are identical if we stipulate that the weights of the feedforward network are identical at each layer (i.e. w

We show that deep learning network derivatives have a low-rank structure, and this structure allows us to use second-order derivative information to calculate learning rates adaptively and in a computationally feasible manner.

Represent the short summary for finding the corresponding scientific article (long)

We propose a hypothesis for why gradient descent generalizes based on how per-example gradients interact with each other.


Question: Represent the scientific article to find a short TLDR

Question: The worst-case training principle that minimizes the maximal adversarial loss, also known as adversarial training (AT), has shown to be a state-of-the-art approach for enhancing adversarial robustness against norm-ball bounded input perturbations. Nonetheless, min-max optimization beyond the purpose of AT has not been rigorously explored in the research of adversarial attack and defense. In particular, given a set of risk sources (domains), minimizing the maximal loss induced from the domain set can be reformulated as a general min-max problem that is different from AT. Examples of this general formulation include attacking model ensembles, devising universal perturbation under multiple inputs or data transformations, and generalized AT over different types of attack models. We show that these problems can be solved under a unified and theoretically principled min-max optimization framework. We also show that the self-adjusted domain weights learned from our method provides a means to explain the difficulty level of attack and defense over multiple domains. Extensive experiments show that our approach leads to substantial performance improvement over the conventional averaging strategy. Training a machine learning model that is capable of assuring its worst-case performance against all possible adversaries given a specified threat model is a fundamental yet challenging problem, especially for deep neural networks (DNNs) (; ;). A common practice to train an adversarially robust model is based on a specific form of min-max training, known as adversarial training (AT) , where the minimization step learns model weights under the adversarial loss constructed at the maximization step in an alternative training fashion. On datasets such as MNIST and CIFAR-10, AT has achieved the state-of-the-art defense performance against p -norm-ball input perturbations (b). Motivated by the success of AT, one follow-up question that naturally arises is: Beyond AT, can other types of min-max formulation and optimization techniques advance the research in adversarial robustness? In this paper, we give an affirmative answer corroborated by the substantial performance gain and the ability of self-learned risk interpretation using our proposed min-max framework on several tasks for adversarial attack and defense. We demonstrate the utility of a general formulation for min-max optimization minimizing the maximal loss induced from a set of risk sources (domains). Our considered min-max formulation is fundamentally different from AT, as our maximization step is taken over the probability simplex of the set of domains. Moreover, we show that many problem setups in adversarial attacks and defenses can in fact be reformulated under this general min-max framework, including attacking model ensembles (Tramèr et al., 2018;), devising universal perturbation to input samples or data transformations , and generalized AT over multiple types of threat models (Tramèr & ;). However, current methods for solving these tasks often rely on simple heuristics (e.g., Work Recent studies have identified that DNNs are highly vulnerable to adversarial manipulations in various applications (; ; ; ; ; ; ; ; a;), thus leading to an arms race between adversarial attacks (; b; ; a; ; b;) and defenses (; b; ; ;). One intriguing property of adversarial examples is the transferability across multiple domains (; Tramèr et al., 2017; ;), which indicates a more challenging yet promising research directiondevising universal adversarial perturbations over model ensembles (Tramèr et al., 2018;), input samples (; ;) and data transformations (b; ;). However, current approaches suffer from a significant performance loss for resting on the uniform averaging strategy. We will compare these works with our min-max method in Sec. 4. As a natural extension following min-max attack, we study the generalized AT under multiple perturbations (Tramèr & ; ; ;). Finally, our min-max framework is adapted and inspired by previous literature on robust learning over multiple domains (; ; ; 2019a). We begin by introducing the principle of robust learning over multiple domains and its connection to a specialized form of min-max optimization. We then show that the ing min-max formulation fits into various attack settings for adversarial exploration: a) ensemble adversarial attack, b) universal adversarial perturbation and c) robust perturbation over data transformations. Finally, we propose a generalized adversarial training (AT) framework under mixed types of adversarial attacks to improve model robustness. Consider K loss functions {F i (v)} (each of which is defined on a learning domain), the problem of robust learning over K domains can be formulated as (; ;) where v and w are optimization variables, V is a constraint set, and P denotes the probability simplex P = {w | 1 T w = 1, w i ∈, ∀i}. Since the inner maximization problem in is a linear function of w over the probabilistic simplex, problem is thus equivalent to where [K] denotes the integer set {1, 2, . . ., K}. Benefit and computation challenge of Compared to multi-task learning in a finite-sum formulation which minimizes K losses on average, problem provides consistently robust worst-case performance across all domains. This can be explained from the epigraph form of, minimize v∈V,t t, subject to where t is an epigraph variable that provides the t-level robustness at each domain. Although the min-max problem offers a great robustness interpretation as in, solving it becomes more challenging than solving the finite-sum problem. It is clear from that the inner maximization problem of always returns the one-hot value of w, namely, w = e i, where e i is the ith standard basis vector, and i = arg max i {F i (v)}. The one-hot coding reduces the generalizability to other domains and induces instability of the learning procedure in practice. Such an issue is often mitigated by introducing a strongly concave regularizer in the inner maximization step . Regularized problem formulation Spurred by , we penalize the distance between the worst-case loss and the average loss over K domains. This yields where γ > 0 is a regularization parameter. As γ → 0, problem is equivalent to. By contrast, it becomes the finite-sum problem when γ → ∞ since w → 1/K. In this sense, the trainable w provides an essential indicator on the importance level of each domain. The larger the weight is, the more important the domain is. We call w domain weights in this paper. We next show how the principle of robust learning over multiple domains can fit into various settings of adversarial attack and defense problems. The general goal of adversarial attack is to craft an adversarial example x = x 0 + δ ∈ R d to mislead the prediction of machine learning (ML) or deep learning (DL) systems, where x 0 denotes the natural example with the true label t 0, and δ is known as adversarial perturbation, commonly subject to given small number. Here the p norm enforces the similarity between x and x 0, and the input space of ML/DL systems is normalized to d. Ensemble attack over multiple models Consider K ML/DL models, the goal is to find robust adversarial examples that can fool all K models simultaneously. In this case, the notion of'domain' in is specified as'model', and the objective function F i in signifies the attack loss f (δ; x 0, y 0, M i) given the natural input (x 0, y 0) and the model M i. Thus, problem becomes where w encodes the difficulty level of attacking each model. and a single model M, our goal is to find the universal perturbation δ so that all the corrupted K examples can fool M. In this case, the notion of'domain' in is specified as'example', and problem becomes where different from, w encodes the difficulty level of attacking each example. Adversarial attack over data transformations Consider K categories of data transformation {p i}, e.g., rotation, lightening, and translation (a), our goal is to find the adversarial attack that is robust to data transformations. In this case, the notion of'domain' in is specified as'data transformer', and problem becomes where E t∼pi [f (t(x 0 +δ); y 0, M)] denotes the attack loss under the distribution of data transformation p i, and w encodes the difficulty level of attacking each type of transformed example x 0. Conventional AT is restricted to a single type of norm-ball constrained adversarial attack . For example, AT under ∞ attack yields minimize where θ ∈ R n denotes model parameters, δ denotes -tolerant ∞ attack, and f tr (θ, δ; x, y) is the training loss under perturbed examples {(x + δ, y)}. However, there possibly exist blind attacking spots across multiple types of adversarial attacks so that AT under one attack would not be strong enough against another attack . Thus, an interesting question is how to generalize AT under multiple types of adversarial attacks. One possible way is to use the finite-sum formulation where δ i ∈ X i is the ith type of adversarial perturbation defined on X i, e.g., different p attacks. Moreover, one can map'attack type' to'domain' considered in. We then perform AT against the strongest adversarial attack across K attack types in order to avoid blind attacking spots. That is, upon defining F i (θ):= maximize δi∈Xi f tr (θ, δ i ; x, y), we solve the problem of the form, minimize In fact, problem is in the min-max-max form, however, Lemma 1 shows that problem can be further simplified to the min-max form. Lemma 1. Problem is equivalent to where w ∈ R K represent domain weights, and P has been defined in. Proof: see Appendix A. Similar to, a strongly concave regularizer −γ/2 w − 1/K 2 2 can be added into the inner maximization problem of, which can boost the stability of the learning procedure and strike a balance between the max and the average attack performance. However, solving problem and its regularized version is more complicated than since the inner maximization involves both domain weights w and adversarial perturbations {δ i}. We finally remark that there was an independent work (Tramèr &) which also proposed the formulation for AT under multiple perturbations. However, what we propose here is the regularized formulation of. As will be evident later, the domain weights w in our formulation have strong interpretability, which learns the importance level of different attacks. Most significantly, our work has different motivation from (Tramèr &), and our idea applies to not only AT but also attack generation in Sec. 2.2. In this section, we delve into technical details on how to efficiently solve problems of robust adversarial attacks given by the generic form and problem for generalized AT under mixed types of adversarial attacks. We propose the alternating one-step projected gradient descent (APGD) method (Algorithm 1) to solve problem. For clarity, we repeat problem under the adversarial perturbation δ and its constraint set X defined in Sec. 2.2, We show that at each iteration, APGD takes only one-step PGD for outer minimization and one-step projected gradient ascent for inner maximization (namely, PGD for its negative objective function). We also show that each alternating step has a closed-form expression, and the main computational complexity stems from computing the gradient of the attack loss w.r.t. the input. Therefore, APGD is computationally efficient like PGD, which is commonly used for design of conventional single p -norm based adversarial attacks . Outer minimization Considering w = w (t−1) and, we perform one-step PGD to update δ at iteration t, where proj(·) denotes the Euclidean projection operator, i.e., proj X (a) = arg min x∈X x − a 2 2 at the point a, α > 0 is a given learning rate, and ∇ δ denotes the first order gradient w.r.t. δ. In, the projection operation becomes the key to obtain the closed-form of the updating rule. Recall from Sec. 2.2 that X = {δ| δ p ≤,č ≤ δ ≤ĉ}, where p ∈ {0, 1, 2, ∞}, andč = −x 0 andĉ = 1 − x 0 (implyingč ≤ 0 ≤ĉ). If p = ∞, then the projection function becomes the clip function. However, when p ∈ {0, 1, 2}, the closed-form of projection operation becomes non-trivial. In Proposition 1, we derive the solution of proj X (a) under different p norms. Proposition 1. Given a point a ∈ R d and a constraint set X = {δ| δ p ≤,č ≤ δ ≤ĉ}, the Euclidean projection δ * = proj X (a) has a closed-form solution when p ∈ {0, 1, 2}. Proof: See Appendix B. Inner maximization By fixing δ = δ (t) and letting ψ(w): in problem, we then perform one-step PGD (w.r.t. −ψ) to update w, where β > 0 is a given learning rate, In, the second equality holds due to the closed-form of projection operation onto the probabilistic simplex P , where (·) + denotes the elementwise nonnegative operator, i.e., (x) + = max{0, x}, and µ is the root of the equation 1, the root µ exists within the interval [min i {b i} − 1/K, max i {b i} − 1/K] and can be found via the bisection method . Convergence analysis We remark that APGD follows the gradient primal-dual optimization framework (a), and thus enjoys the same optimization guarantees. In Theorem 1, we demonstrate the convergence rate of Algorithm 1 for solving problem. Theorem 1. (inherited from primal-dual min-max optimization) Suppose that in problem F i (δ) has L-Lipschitz continuous gradients, and X is a convex compact set. Given learning rates α ≤ We next propose the alternating multi-step projected gradient descent (AMPGD) method to solve the regularized version of problem, which is repeated as follows Algorithm 2 AMPGD to solve problem given w (t−1) and δ (t−1), perform SGD to update θ (t) 4: given θ (t), perform R-step PGD to update w (t) and δ (t) Problem is in a more general non-convex non-concave min-max setting, where the inner maximization involves both domain weights w and adversarial perturbations {δ i}. It was shown in that the multi-step PGD is required for inner maximization in order to approximate the near-optimal solution. This is also in the similar spirit of AT , which executed multi-step PGD attack during inner maximization. We summarize AMPGD in Algorithm 2. At step 4 of Algorithm 2, each PGD step to update w and δ can be decomposed as where let w 1:= w (t−1) and δ. Here the subscript t represents the iteration index of AMPGD, and the subscript r denotes the iteration index of R-step PGD. Clearly, the above projection operations can be derived for closed-form expressions through and Lemma 1. To the best of our knowledge, it is still an open question to build theoretical convergence guarantees for solving the general non-convex non-concave min-max problem like, except the work which proposed O(1/T) convergence rate if the objective function satisfies Polyak-Łojasiewicz conditions . Improved robustness via diversified p attacks. It was recently shown in that the diversity of individual neural networks improves adversarial robustness of an ensemble model. Spurred by that, one may wonder if the promotion of diversity among p attacks is beneficial to adversarial robustness? We measure the diversity between adversarial attacks through the similarity between perturbation directions, namely, input gradients {∇ δi f tr (θ, δ i ; x, y)} i in. We find that there exists a strong correlation between input gradients for different p attacks. Thus, we propose to enhance their diversity through the orthogonality-promoting regularizer used for encouraging diversified prediction of ensemble models in , where G ∈ R d×K is a d × K matrix, each column of which corresponds to a normalized input gradient ∇ δi f tr (θ, δ i ; x, y) for i ∈ [K], and h(θ, {δ i}; x, y) reaches the maximum value 0 as input gradients become orthogonal. With the aid of, we modify problem to minimize The rationale behind is that the adversary aims to enhance the effectiveness of attacks from diversified perturbation directions (inner maximization), while the defender robustifies the model θ, which makes diversified attacks less effective (outer minimization). In this section, we first evaluate the proposed min-max optimization strategy on three attack tasks. We show that our approach leads to substantial improvement compared with state-of-the-art attack methods such as ensemble PGD and expectation over transformation (EOT) (b; ; a). We next demonstrate the effectiveness of the generalized AT for multiple types of adversarial perturbations. We show that the use of trainable domain weights in problem can automatically adjust the risk level of different attacks during the training process even if the defender lacks prior knowledge on the strength of these attacks. We also show that the promotion of diversity of p attacks help improve adversarial robustness further. We thoroughly evaluate our APGD/AMPGD algorithm on MNIST and CIFAR-10. A set of diverse image classifiers (denoted from Model A to Model H) are trained, including multi-layer perceptrons (Most current works play a min-max game from a defender's perspective, i.e., adversarial training. However, we show the great strength of min-max optimization also lies at the side of attack generation. Note that problem formulations- are applicable to both untargeted and targeted attack. Here we focus on the former setting and use C&W loss function . The details of crafting adversarial examples are available in Appendix C.2. Ensemble attack over multiple models We craft adversarial examples against an ensemble of known classifiers. The work (, 5th place at CAAD-18) proposed an ensemble PGD attack, which assumed equal importance among different models, namely, w i = 1/K in problem. Throughout this task, we measure the attack performance via ASR all -the attack success rate (ASR) of fooling model ensembles simultaneously. Compared to the ensemble PGD attack , our approach in 40.79% and 17.48% ASR all improvement averaged over different p -norm constraints on MNIST and CIFAR-10, respectively. In what follows, we provide more detailed experiment and analysis. In Table 1, we show that our min-max APGD significantly outperforms ensemble PGD in ASR all. Taking ∞ -attack on MNIST as an example, our min-max attack leads to a 90.16% ASR all, which largely outperforms 48.17% (ensemble PGD). The reason is that Model C, D are more difficult to attack, which can be observed from their higher test accuracy on adversarial examples. As a , although the adversarial examples crafted by assigning equal weights over multiple models are able to attack {A, B} well, they achieve a much lower ASR (i.e., 1 -Acc) in {C, D}. By contrast, APGD automatically handles the worst case {C, D} by slightly sacrificing the performance on {A, B}: 31.47% averaged ASR improvement on {C, D} versus 0.86% degradation on {A, B}. More on CIFAR-10 and more complicated DNNs (e.g., GoogLeNet) are provided in Appendix D. Lastly, we highlight that tracking domain weights w provides us novel insights for model robustness and understanding attack procedure. From our theory, a model with higher robustness always corresponds to a larger w because its loss is hard to attack and becomes the "worst" term. This hypothesis can be verified empirically. According to Figure 1c, we have w c > w d > w a > w bindicating a decrease in model robustness for C, D, A and B, which is exactly verified by Acc C > Acc D > Acc A > Acc B in Table 1 (∞ -norm). Universal perturbation over multiple examples We evaluate APGD in universal perturbation on MNIST and CIFAR-10, where 10,000 test images are randomly divided into equal-size groups (containing K images per group) for universal perturbation. We measure two types of ASR (%), ASR avg and ASR gp. Here the former represents the ASR averaged over all images in all groups, and the latter signifies the ASR averaged over all groups but a successful attack is counted under a more restricted condition: images within each group must be successfully attacked simultaneously by universal perturbation. When K = 5, our approach achieves 42.63% and 35.21% improvement over the averaging strategy under MNIST and CIFAR-10, respectively. In Table 2, we compare the proposed min-max strategy (APGD) with the averaging strategy on the attack performance of generated universal perturbations. As we can see, our method always achieves higher ASR gp for different values of K. The universal perturbation generated from APGD can successfully attack'hard' images (on which the average-based PGD attack fails) by self-adjusting domain weights, and thus leads to a higher ASR gp. Besides, the min-max universal perturbation also offers interpretability of "image robustness" by associating domain weights with image visualization; see Figure A9 and A10 (Appendix F) for an example in which the large domain weight corresponds to the MNIST letter with clear appearance (e.g., bold letter). Robust adversarial attack over data transformations EOT (a) achieves stateof-the-art performance in producing adversarial examples robust to data transformations. From, we could derive EOT as a special case when the weights satisfy w i = 1/K (average case). For each input sample (ori), we transform the image under a series of functions, e.g., flipping horizontally and γ = 4. (flh) or vertically (flv), adjusting brightness (bri), performing gamma correction (gam) and cropping (crop), and group each image with its transformed variants. Similar to universal perturbation, ASR avg and ASR gp are reported to measure the ASR over all transformed images and groups of transformed images (each group is successfully attacked signifies successfully attacking an example under all transformers). In Table 3, compared to EOT, our approach leads to 9.39% averaged lift in ASR gp over given models on CIFAR-10 by optimizing the weights for various transformations. Due to limited space, we leave the details of transformers in Append C.3 and the under randomness (e.g., flipping images randomly w.p. 0.8; randomly clipping the images at specific range) in Appendix D. Compared to vanilla AT, we show the generalized AT scheme produces models robust to multiple types of perturbation, thus leads to stronger "overall robustness". We measure the training performance using two types of Acc (%): Acc max adv and Acc avg adv, where Acc max adv denotes the test accuracy over examples with the strongest perturbation (∞ or 2), and Acc avg adv denotes the averaged test accuracy over examples with all types of perturbations (∞ and 2). Moreover, we measure the overall worst-case robustness S in terms of the area under the curve'Acc max adv vs.' (see Figure 3b). In Table 4, we present the test accuracy of MLP in different training schemes: a) natural training, b) single-norm: vanilla AT (∞ or 2), c) multi-norm: generalized AT (avg and min max), and d) generalized AT with diversity-promoting attack regularization (DPAR, λ = 0.1 in problem). If the adversary only performs single-type attack, training and testing on the same attack type leads to the best performance (diagonal of ∞ -2 block). However, when facing ∞ and 2 attacks simultaneously, multi-norm generalized AT achieves better Acc max adv and Acc avg adv than single-norm AT. In particular, the min-max strategy slightly outperforms the averaging strategy under multiple perturbation norms. Figure A2 (Appendix E). DPAR further boosts the adversarial test accuracy, which implies that the promotion of diversified p attacks is a beneficial supplement to adversarial training. In Figure 3, we offer deeper insights on the performance of generalized AT. During the training procedure we fix ∞ (for ∞ attack during training) as 0.2, and change 2 from 0.2 to 5.6 (∞ × √ d) so that the ∞ and 2 balls are not completely overlapped . In Figure 3a, as 2 increases, 2 -attack becomes stronger so the corresponding w also increases, which is consistent with min-max spirit -defending the strongest attack. We remark that min max or avg training does not always lead to the best performance on Acc max adv and Acc avg adv, especially when the strengths of two attacks diverge greatly (see Table A8). This can be explained by the large overlapping between ∞ and 2 balls (see Figure A3). However, Figure 3b and 3c show that AMPGD is able to achieve a rather robust model no matter how changes (red lines), which empirically verifies the effectiveness of our proposed training scheme. In terms of the area-under-the-curve measure S, AMPGD achieves the highest worst-case robustness: 6.27% and 17.64% improvement compared to the vanilla AT with ∞ and 2 attacks. Furthermore, we show in Figure A4a that our min-max scheme leads to faster convergence than the averaging scheme due to the benefit of self-adjusted domain weights. In this paper, we propose a general min-max framework applicable to both adversarial attack and defense settings. We show that many problem setups can be re-formulated under this general framework. Extensive experiments show that proposed algorithms lead to significant improvement on multiple attack and defense tasks compared with previous state-of-the-art approaches. In particular, we obtain 17.48%, 35.21% and 9.39% improvement on attacking model ensembles, devising universal perturbation to input samples, and data transformations under CIFAR-10, respectively. Our minmax scheme also generalizes adversarial training (AT) for multiple types of adversarial attacks, attaining faster convergence and better robustness compared to the vanilla AT and the average strategy. Moreover, our approach provides a holistic tool for self-risk assessment by learning domain weights. where w ∈ R K represent domain weights, and P has been defined in. Similar to, problem is equivalent to Recall that F i (θ):= maximize δi∈Xi f tr (θ, δ i ; x, y), problem can then be written as According to proof by contradiction, it is clear that problem is equivalent to B PROOF OF PROPOSITION 1 Proposition 1. Given a point a ∈ R d and a constraint set X = {δ| δ p ≤,č ≤ δ ≤ĉ}, the Euclidean projection δ * = proj X (a) has the closed-form solution when p ∈ {0, 1, 2}. 1) If p = 1, then δ * is given by where x i denotes the ith element of a vector x; P [či,ĉi] (·) denotes the clip function over the interval where λ 2 ∈ (0, a 2 / − 1] is the root of 3) If p = 0 and ∈ N +, then δ * is given by where [η] denotes the -th largest element of η, and δ i = P [či,ĉi] (a i). 1 norm When we find the Euclidean projection of a onto the set X, we solve where I [č,ĉ] (·) is the indicator function of the set [č,ĉ]. The Langragian of this problem is The minimizer δ * minimizes the Lagrangian, it is obtained by elementwise soft-thresholding where x i is the ith element of a vector x, P [či,ĉi] (·) is the clip function over the interval The primal, dual feasibility and complementary slackness are, where λ 1 is given by the root of the equation Bisection method can be used to solve the above equation for λ 1, starting with the initial interval (0, max 2 norm When we find the Euclidean projection of a onto the set X, we solve where I [č,ĉ] (·) is the indicator function of the set [č,ĉ]. The Langragian of this problem is The minimizer δ * minimizes the Lagrangian, it is The primal, dual feasibility and complementary slackness are λ2+1 a i, where λ 2 is given by the root of the equation Bisection method can be used to solve the above equation for λ 2, starting with the initial interval (0, 2 > 2 in this case, and 0 norm For 0 norm in X, it is independent to the box constraint. So we can clip a to the box constraint first, which is δ i = P [či,ĉi] (a i), and then project it onto 0 norm. We find the additional Euclidean distance of every element in a and zero after they are clipped to the box constraint, which is It can be equivalently written as To derive the Euclidean projection onto 0 norm, we find the -th largest element in η and call it [η]. We keep the elements whose corresponding η i is above or equals to -th, and set rest to zeros. The closed-form solution is given by.1) that the problem is convex and the solution can be derived using KKT conditions. However, Proposition 1 in our paper is different from (, Proposition 4.1). First, we place p norm as a hard constraint rather than minimizing it in the objective function. This difference will make our Lagrangian function more involved with a newly introduced nonnegative Lagrangian multiplier. Second, the problem of our interest is projection onto the intersection of box and p constraints. Such a projection step can then be combined with an attack loss (no need of linearization) for generating adversarial examples. Third, we cover the case of 0 norm. C. For the last four models, we use the exact same architecture as original papers and evaluate them only on CIFAR-10 dataset. The details for model architectures are provided in Table A1. For compatibility with our framework, we implement and train these models based on the strategies adopted in pytorch-cifar classifiers for 50 epochs with Adam and a constant learning rate of 0.001. For CIFAR-10 classifers, the models are trained for 250 epochs with SGD (using 0.8 nesterov momentum, weight decay 5e −4). The learning rate is reduced at epoch 100 and 175 with a decay rate of 0.1. The initial learning rate is set as 0.01 for models {A, B, C, D, H} and 0.1 for {E, F, G}. Note that no data augmentation is employed in the training. with a confidence parameter κ = 50. Cross-entropy loss is also supported in our implementation. The adversarial examples are generated by 20-step PGD/APGD unless otherwise stated (e.g., 50 steps for ensemble attacks). Note that proposed algorithms are robust and will not be affected largely by the choices of hyperparameters (α, β, γ). In consequence, we do not finely tune the parameters on the validation set. Specifically, The learning rates α, β and regularization factor γ for Table 1 Moreover, both deterministic and stochastic transformations are considered in our experiments. In particular, Table 3 and Table A6 are deterministic settings -rot: rotating images 30 degree clockwise; crop: cropping images in the center (0.8 × 0.8) and resizing them to 32 × 32; bri: adjusting the brightness of images with a scale of 0.1; gam: performing gamma correction with a value of 1.3. Differently, in Table A5, we introduce randomness for drawing samples from the distribution -rot: rotating images randomly from -10 to 10 degree; crop: cropping images in the center randomly (from 0.6 to 1.0); other transformations are done with a probability of 0.8. In experiments, we adopt tf.image API 7 for processing the images. Table A3 shows the performance of average (ensemble) and min-max (APGD) strategies for attacking model ensembles. Our min-max approach in 15.69% averaged improvement on ASR all over models {A, E, F, H} on CIFAR-10., γ = 6. The attack iteration for APGD is set as 50. Opt. To further demonstrate the effectiveness of self-adjusted weighting factors in proposed min-max framework, we compare with heuristic weighting schemes in Table A4 Table A4 shows that our min-max approach outperforms all static heuristic weighting schemes by a large margin. Specifically, our min-max APGD also achieves significant improvement compared to w static setting, where the converged optimal weights are statically (i.e., invariant w.r.t different images and attack procedure) adopted. It again verifies the benefits of proposed min-max approach by automatically learning the weights for different examples during the process of ensemble attack generation (see Figure 1c). Table A5 and A6 compare the performance of average (EOT Athalye et al. (2018a) ) and min-max (APGD) strategies. Our approach in 4.31% and 8.22% averaged lift over four models {A, B, C, D} on CIFAR-10 under given stochastic and deterministic transformation sets. and γ = 10. To further explore the utility of quadratic regularizer on the probability simplex in proposed min-max framework, we conducted sensitivity analysis on γ and show how the proposed regularization affects the eventual performance (Figure A1) taking ensemble attack as an example. The experimental setting is the same as Table 1 except for altering the value of γ from 0 to 10. Figure A1 shows that too small or too large γ leads to relative weak performance due to the unstable convergence and penalizing too much for average case. When γ is around 4, APGD will achieve the best performance so we adopted this value in the experiments (Table 1). Moreover, when γ → ∞, the regularizer term dominates the optimization objective and it becomes the average case. Figure A2 presents "overall robustness" comparison of our min-max generalized AT scheme and vanilla AT with single type of attacks (∞ and 2) on MNIST (LeNet). Similarly, our min-max training scheme leads to a higher "overall robustness" measured by S. In practice, due to the lacking knowledge of the strengths/types of the attacks used by adversaries, it is meaningful to enhance "overall robustness" of models under the worst perturbation (Acc max adv). Specifically, our min-max generalized AT leads to 6.27% and 17.63% improvement on S compared to single-type AT with ∞ and 2 attacks. Furthermore, weighting factor w of the probability simplex helps understand the behavior of AT under mixed types of attacks. Our AMPGD algorithm will adjust w automatically according to the min-max principle -defending the strongest attack. In Figure A2a, as 2 increases, 2 -attack becomes stronger so its corresponding w increases as well. When 2 ≥ 2.5, 2 -attack dominates the adversarial training process. That is to say, our AMPGD algorithm will put more weights on stronger attacks even if the strengths of attacks are unknown, which is a meritorious feature in practice. also propose a variant of adversarial training to defend universal perturbations over multiple images. To produce universal perturbations, they propose uSGD to conduct gradient descent on the averaged loss of one-batch images. In consequence, their approach can be regarded as a variant of our generalized AT in average case. The difference is that they do AT across multiple adversarial images under universal perturbation rather than mixed p -norm perturbations. We added UAT as one of our defense baselines in Table A7. The universal perturbation is generated by uSGD (∞ norm, = 0.3) with a batch size of 128 following. We find that a) our proposed approach outperforms UAT under per-image p attacks. Taking A7a as an example, our avg and min max generalized AT (with DPAR) in average 17.85% and 17.97% improvement in adversarial test accuracy (ATA), b) our approach has just 3.72% degradation in ATA when encountering universal attacks, and c) both methods yield very similar normal test accuracy. It is not surprising that our average and min-max training schemes can achieve better overall robustness while maintaining competitive performance on defending universal perturbation. This is because the defensed model is trained under more general (p norm) and diversity promoted perturbations. As a , proposed generalized AT is expected to obtain better overall robustness and higher transferability as shown in Table 4 and A7. As reported in Sec. 4.2, our min-max generalized AT does not always in the best performance on the success rate of defending the worst/strongest perturbation (Acc max adv) for given (∞, 2) pair, especially when the strengths of two attacks diverge greatly (e.g., for ∞ and 2 attacks are 0.2 and 0.5). In what follows, we provide explanation and analysis about this finding inspired by recent work. and inside 2 ball (right, red area). In particular, the red (blue) area in (a) (or (b)) represents the percentage of adversarial examples crafted by ∞ attack that also belong to 2 (∞) ball. We generate adversarial examples on 10,000 test images for each attack. (c): Average p norm of adversarial examples as a function of perturbation magnitude 2. The top (bottom) side represents the 2-norm (∞) of the adversarial examples generated by ∞ attack as 2 for generalized AT increases. Note that the same as the AT procedure is used while attacking trained robust models. Figure A3 shows the real overlap of ∞ and 2 norm balls in adversarial attacks for MLP model on MNIST. Ideally, if 2 satisfies ∞ < 2 < ∞ × √ d, ∞ and 2 balls will not cover each other completely. In other words, AT with ∞ and 2 attacks cannot interchange with each other. However, the real range of 2 for keeping 2 and ∞ balls intersected is not, because crafted adversarial examples are not uniformly distributed in p -norm balls. In Figure A3b, 99.98% adversarial examples devising using 2 attack are also inside ∞ ball, even if 0.2 < 2 = 0.5 < 5.6. In consequence, AT with ∞ attack is enough to handle 2 -attack in overwhelming majority cases, which in better performance than min-max optimization (Table A8a). Figure A3c presents the average p distance of adversarial examples with 2 increasing. The average 2 -norm (green line) of adversarial examples generated by ∞ attack remains around 2.0 with a slight rising trend. This is consistent to our setting -fixing 2 as 0.2. It also indicates model robustness may effect the behavior of attacks -as 2 increases, robustly trained MLP model becomes more robust against 2 examples, so the ∞ attacker implicitly increases 2 norm to attack the model more effectively. On the other hand, the average ∞ -norm increases substantially as 2 increases from 0.5 to 2.5. When 2 arriving at 0.85, the average ∞ norm gets close to 0.2, so around half adversarial examples generated by 2 -attack are also inside ∞ balls, which is consistent with Table A3b. Figure A4 shows the learning curves of model A under different AT schemes, where two setting are plotted: (a) (∞, 2) = (0.2, 0.5); (b) (∞, 2) = (0.2, 2.0). Apart from better worst-case robustness shown in Table A8, our min-max generalized AT leads to a faster convergence compared to average-based AT, especially when the strengths of two attacks diverge greatly. For instance, when 2 = 0.5 (Figure A4a Tracking domain weight w of the probability simplex from our algorithms is an exclusive feature of solving problem 1. In Sec. 4, we show the strength of w in understanding the procedure of optimization and interpreting the adversarial robustness. Here we would like to show the usage of w in measuring "image robustness" on devising universal perturbation to multiple input samples. Table A9 and A10 show the image groups on MNIST with weight w in APGD and two metrics (distortion of 2-C&W, minimum for ∞ -PGD) of measuring the difficulty of attacking single images. The binary search is utilized to searching for the minimum perturbation. Although adversaries need to consider a trade-off between multiple images while devising universal perturbation, we find that weighting factor w in APGD is highly correlated under different p norms. Furthermore, w is also highly related to minimum distortion required for attacking a single image Table A8: Adversarial training of MNIST models with single attacks (∞ and 2) and multiple attacks (avg. and min max). During the training process, the perturbation magnitude ∞ is fixed as 0.2, and 2 are changed from 0.5 to 3.0 with a step size of 0.5. For min-max scheme, the adversarial examples are crafted using 20-step ∞-APGD with α = 1 6, β = 1 50 and γ = 4. The ratio of adversarial and benign examples in adversarial training is set as 1.0. For diversity-promoting attack regularizer (DPAR) in generalized AT, the hyperparameter λ = 0.1. (a) (∞, 2) = (0.2, 0.5) successfully. It means the inherent "image robustness" exists and effects the behavior of generating universal perturbation. Larger weight w usually indicates an image with higher robustness (e.g., fifth 'zero' in the first row of Table A9), which usually corresponds to the MNIST letter with clear appearance (e.g., bold letter).

Represent the short summary for finding the corresponding scientific article (long)

A unified min-max optimization framework for adversarial attack and defense

Represent the TLDR for finding the corresponding scientific article.

In this paper, a new method we call Centered Initial Attack (CIA) is provided. It insures by construction the maximum perturbation to be smaller than a threshold fixed beforehand, without the clipping process.