Question: Represent the article for finding a single-sentence TLDR

Question: There were many attempts to explain the trade-off between accuracy and adversarial robustness. However, there was no clear understanding of the behaviors of a robust classifier which has human-like robustness. We argue why we need to consider adversarial robustness against varying magnitudes of perturbations not only focusing on a fixed perturbation threshold, why we need to use different method to generate adversarially perturbed samples that can be used to train a robust classifier and measure the robustness of classifiers and why we need to prioritize adversarial accuracies with different magnitudes. We introduce Lexicographical Genuine Robustness (LGR) of classifiers that combines the above requirements. We also suggest a candidate oracle classifier called "Optimal Lexicographically Genuinely Robust Classifier (OLGRC)" that prioritizes accuracy on meaningful adversarially perturbed examples generated by smaller magnitude perturbations. The training algorithm for estimating OLGRC requires lexicographical optimization unlike existing adversarial training methods. To apply lexicographical optimization to neural network, we utilize Gradient Episodic Memory (GEM) which was originally developed for continual learning by preventing catastrophic forgetting. Even though deep learning models have shown promising performances in image classification tasks, most deep learning classifiers mis-classify imperceptibly perturbed images, i.e. adversarial examples. This vulnerability can occur even when the adversarial attacks were applied before they print the images, and the printed images were read through a camera. That shows real-world threats of classifiers can exist. In addition, adversarial examples for a classifier can be transferable to other models. This transferability of adversarial examples enables attackers to exploit a target model with limited access to the target classifier. This kinds of attacks is called black-box attacks. An adversarially perturbed sample refers to the of the perturbation (adversary generation) methods that has increased adversarial loss usually starting from an original sample. It is important to notice that an adversarially perturbed sample of a classifier may not be an adversarial example, which will be explained later in subsection 1.1. It can be just a non-adversarial perturbed sample (see Figure 4). Adversary generation methods try to effectively increase adversarial loss using the available information of the target classifier. Methods to generate adversarially perturbed samples include Fast Gradient Sign Method (FGSM), Basic Iterative Method (BIM), Projected Gradient Descent (PGD), Distributionally Adversarial Attack (DAA) and Interval Attack. We will use the following terminology for the following paragraphs unless we specify otherwise. x is an original sample, y is corresponding label for x, is the perturbation norm, sign indicates the element-wise sign function and L(θ, x, y) is the loss of the classifier parameterized by θ. Fast Gradient Sign Method (FGSM) generates the adversarial x F GSM with the following formula. x F GSM = x + sign [∇ x L(θ, x, y)]. FGSM was suggested from the hypothesis that linear behaviors of classifiers are enough to cause adversarial susceptibility of models. The formula was obtained by applying local linearization of the cost function and finding the optimal perturbation. Note that we only show formula for l ∞ norm (max norm) attacks, but we can easily get the formula for other attacks when we replace the sign function with identity function or others. In order to get the strongest attacks using first order information of the models, Projected gradient descent (PGD) generates the adversarial x P GD by applying iterative steps like the following., y) where x + S is the set of allowed perturbation region for sample x that is limited by, x is the random starting points in x + S, x+S [] means the projection on x + S and α is the step size. Note that we also only show formula for l ∞ norm attacks. Basic Iterative Method (BIM) use the same iterative steps with PGD method except that it start from a fixed starting point, i.e. x = x for BIM. Adversarial training was developed to avoid adversarial vulnerability of a classifier. It tries to reduce the weighted summation of standard loss (empirical risk) E [L(θ, x, y)] and adversarial loss E [L(θ, x, y)], i.e. αE [L(θ, x, y)] + (1 − α)E [L(θ, x, y)] where α is a hyperparameter for adversarial training, and x is an adversarially perturbed sample from x with x − x ≤. (Usually, α = 0.5 is used for adversarial training.) By considering both standard and adversarially perturbed samples, adversarial training try to increase accuracies on both clean 3 and adversarially perturbed samples. In the literatures on adversarial training, inner maximization of a classifier refers to generating adversarial attacks, i.e. generating adversarially perturbed samples x * that maximally increase the loss. And outer minimization refers to minimizing the adversarial loss of the model. Madry et al. explained that inner maximization and outer minimization of the loss can train models that are robust against adversarial attacks. However, adversarial training has shown some issues. As some researches on adversarial robustness explained the trade-offs between accuracy on clean data and adversarial robustness, when we used adversarial training, we can get a classifier whose accuracy is lower than using standard (non-adversarial) training method. Also, a research studied samples whose perceptual classes are changed due to perturbation, but not in the model's prediction, what they called "invariance-based adversarial examples". They found that classifiers trained with adversarial training can be more susceptible to invariance-based adversarial examples. We define three properties of human-like classification: human-like classification is robust against varying magnitudes of adversarially perturbed samples and not just on a fixed maximum norm perturbations, when we think about adversarially perturbed samples with increasing magnitudes, a human-like classifier does not consider already considered samples multiple times and human-like classification prioritizes adversarial accuracies with smaller Figure 1: Examples of confusing near image pairs with different classes of MNIST training dataset. The l 2 norms of the pairs are 2.399, 3.100 and 3.131 from left to right. From these examples, we can say the exclusive belongingness assumption may not be realistic. perturbation norm. The objective of this paper is to design and train a classifier whose robustness is more resemblance to robustness of human than a model trained by standard adversarial training. We introduce Lexicographical Genuine Robustness (LGR) of classifiers to combine three properties. Using LGR can prevent the problem of training classifiers with lower accuracy on clean data by considering the third property. LGR also enable to avoid what we named "pseudo adversarial examples" of models which are conceptually similar to invariance-based adversarial examples. From LGR, we introduce a candidate oracle classifier called "Optimal Lexicographically Genuinely Robust Classifier (OLGRC)". We know move on to more precise definition of adversarial examples and the detailed explanation of our ideas. The definition of adversarial example by Biggio et al. was used in many theoretical analysis of adversarial robustness. These analyses showed adversarial examples are inevitable and there is a trade-off between accuracy on clean data and adversarial robustness, i.e. accuracy on adversarially perturbed samples. However, we argue why simply increasing adversarial robustness can get a classifier whose behavior is different from humans. Problem setting 1. In a clean input set X ⊆ R d, let every sample x exclusively belong to one of the classes Y, and their classes will be denoted as c x. A classifier f assigns a class label from Y for each sample x ∈ R d. Assume f is parameterized by θ and L(θ, x, y) is the loss of the classifier provided the input x and the label y ⊆ Y. Note that this exclusive belonging assumption is introduced to simplify the analysis and it can be unrealistic. In a real situation, 1) input information might not be enough to perfectly predict the class, 2) input samples might contain noises which erase class information, 3) some input samples might be better to give non-exclusive class (see Figure 1), or 4) sometimes labels might also contain some noises due to mistakes. Definition 1. Given a clean sample x ∈ X and a maximum permutation norm (threshold), a perturbed sample x is an adversarial example by the definition of Biggio et al. if x − x ≤ and f (x) = c x. Note that some perturbed samples x and some adversarial examples may also belong to X. Although generated by adversary generation methods mentioned in subsection 1.0.1, perturbed samples are not necessarily adversarial examples (see Figure 4). For example, when allowed perturbation norm is too small, predicted class of adversarially perturbed samples x can be c x. We are only focus on the analysis using l p norm for measuring the distance, but the concept of adversarial examples and our analysis are not confined to these metrics. Many ideas in our analysis can might be applied to adversarial examples based on relaxed versions of metrics. Let's consider the classification task on MNIST dataset. We will use the norms calculated by viewing an image input as a (flatten) 784 dimensional vector. The smallest l 2 norm 5 of the image pair for different class on training data is 2.399. However, as the l 2 norm of the nearest image pair for class 6 and 7 on training data is 5.485, when we train a classifier using adversarial training with = 2.399 (one can use half of the minimum distance = 2.399 2 = 1.200 to consider the distance between decision boundary and the samples in different classes, but our explanation doesn't consider that approach), the trained classifier might mis-classifies a perturbed image of digit 6 as an image of digit 7 when perturbation norm is 2.5 (> 2.399) even if such perturbation norm is smaller than the half (2.742) of the expected minimum norm 5.485. Hence, we might want a classifier who is also robust when is larger than 2.399. If we want a classifier which has no adversarial example when = 5.4 < 5.485, we need to have a classifier that outputs original class for every training image and perturbed images when norms of the perturbations are at most 5.4. However, the l 2 norm of the nearest image pair for class 4 and 9 on training data is 2.830 (see Figure 2). What does it mean? As the image on the bottom left can also be considered as an adversarial example perturbed from top left, the classifier needs to classify its class is 9 when it was an original image and its class is 4 when it was an adversarial example. Can we have a classifier with such psychometric power that knows the previous history of images? Do we have such psychometric power? Or, are we not robustness enough even for classifying MNIST data? The more important question we need to ask is "Do we really want such kinds of ability?". And we answer the answer is "No!". This confusion arises from the gap between our intuitive understanding and Biggio et al.'s definition of adversarial examples. (Note that the reason we encounter these kinds of problems is not because we are doing multiclass classification. A similar problem can occur in binary classification problems as shown in Figure 3 .) Even though intuitive definition of adversarial examples are samples which are generated by applying imperceptible perturbations to clean samples, by relying on the Biggio et al.'s definition of adversarial examples, some theoretical analyses tried to analyze the adversarial robustness even when the norms of the adversarial perturbations are big enough so that they can change the perceptual classes of samples. Definition 2. Let us distinguish two kinds of class for a given clean sample x ∈ X and its corresponding perturbed sample x. 5 l∞ norm was more commonly used in the literature as l∞ norm of the perturbation should be large in order to change the perceptual class. However, a classifier trained by adversarial training with l∞ adversary is susceptible to adversarial attacks with l0 or l2 adversary which suggests that we also need to consider l0 and l2 norm robustness. • De facto class: c x of the clean sample x. c x of the perturbed sample x if x ∈ X. De facto class is undefined for some perturbed sample x ∈ X C = R d − X. • De jure class: c x of the clean sample x. c x of the perturbed sample x, i.e. original class of the perturbed example x. Intuitively speaking, de facto class of a sample is current perceptual class of a sample. The name "De jure" can be debatable and confusing, but it follows the tradition of the researchers who consider the original class of a perturbed sample as a legitimate class and try to increase robustness based on that even if the perturbation can change the de facto class. One thing to notice is that we can change the de facto class by perturbating a clean sample x when large perturbation is allowed, but we can't change the de jure class of it. De facto class and de jure class are not dependent on the classifier f. Definition 3. Furthermore, we distinguish two kinds of adversarial example x when x was the original sample of x before the adversarial perturbation. • Pseudo adversarial example: an adversarial example x by definition 1 whose de facto class is different from its de jure class, i.e. c x = c x. • Genuine adversarial example: an adversarial example x by definition 1 whose de facto class is undefined, i.e. x ∈ X C. Note that even though the classifier f determines whether a given perturbed sample x is an adversarial example or not, it doesn't affect whether an adversarial example x is a pseudo adversarial example or genuine adversarial example. For a given sample, the history (whether it was perturbed or not) of a sample will determine whether the sample is a clean sample x or a perturbed sample x. For a perturbed sample x, its de jure class c x and the classifier f will determine whether it is a non-adversarial perturbed sample or an adversarial example. Finally, the existence of de facto class will determine whether an adversarial example is a genuine or pseudo adversarial example. With our definitions, let's consider again the classification task on MNIST dataset (see Figure 2). When we think about adversarial examples for = 5.4, again, the image on the bottom left can be considered as an adversarial example perturbed from the top left image. As its de facto class is 9 no matter it's a clean or adversarial example, it is a pseudo adversarial example of top left image if it was an adversarial example. Do we want our classifier to be robust against pseudo adversarial examples? Short answer to this question is "No, we don't need to.". When we consider the classification process of humans, we do not care about whether a given sample was a clean or perturbed sample, i.e. the previous history of the sample. We only care about the most likely class of the current sample and such class is close to the concept of de facto class. And this principle was commonly used in many visual assessment of adversarial robustness [10, 13, even if some of them follow the definition of adversarial example of Biggio et al.. Let's consider a general situation where a classifier f tries to increase the adversarial robustness for a perturbation norm which is large enough so that the perturbation can change the de facto classes of some samples. In other words, the classifier tries to assign de jure class even for pseudo adversarial examples. This implies that the classifier tries to assign perceptually wrong classes for pseudo adversarial examples who are currently equivalent to clean examples, and this will decrease accuracy on clean data without increasing human-like robustness on these samples. Hence, not only we don't need to increase robustness against pseudo adversarial examples, but also we should avoid increasing robustness against them in order to get a model with human-like robustness (Note that robustness will be calculated by de jure classes of pseudo adversarial examples). Let's compare the training tasks when we only have clean samples and when we only have perturbed samples. Perturbed samples can be derived from clean samples and theoretically they can take any values in their allowed perturbation regions. Because of that perturbed samples have more uncertainty than clean samples. In order words, clean samples have more information than perturbed samples. This observation can lead to a preference to prefer using clean samples when we train a model. When we think about a training task with both clean and perturbed samples, the preference will be correspond to increasing natural accuracy before we consider the accuracy on perturbed samples. This preference can be generalized to a principle that we prioritize the adversarial accuracy on smaller perturbation norm. From the above explanations, we can summarize the properties of human classification or human-like robustness. 1. Human classification is robust against adversarially perturbed samples generated from varying magnitudes of perturbations and not just fixed maximum norm perturbations. 2. The previous history of a sample has no effect in classification. Only the current sample will determine the classification . From this, a human-like classifier avoids assigning de jure class for pseudo adversarial examples. More generally, a human-like classifier avoids considering already considered samples several times. 3. Human classification prioritizes the robustness for smaller perturbation norm than the robustness for larger perturbation norm. The question arising from the second property is "How do we know a given adversarial example is a pseudo adversarial example or genuine adversarial example?". It would be trivial when we know the data distribution and predefined classes for all data like the toy example in section 2. However, in practice, we only have limited training data and hard to know the data distribution. We introduce a method to estimate whether a perturbed sample x has de facto class or not, and thus try to avoid using pseudo adversarial examples for adversarial training and measure the robustness of classifiers. We then combine this with a lexicographical optimization method. Before further diving into the adversarial robustness of classifiers, we give the mathematical definitions of the accuracies. We define natural accuracy and adversarial accuracies for given maximum perturbation norm and exact perturbation norm. Note that 1 is an indicator function which has value 1 if the condition in the bracket holds and value 0 if the condition in the bracket doesn't hold. • Natural accuracy: • (Standard) Adversarial accuracy (by maximum perturbation norm): where adversarially perturbed sample x * = argmax • (Standard) Adversarial accuracy (by exact perturbation norm): where adversarially perturbed sample x * = argmax • Genuine adversarial accuracy (by maximum perturbation norm): where S max = x ∈ X |∃x ∈ X C: x − x ≤ and adversarially perturbed sample x * = argmax • Genuine adversarial accuracy (by exact perturbation norm): where S exact = x ∈ X |∃x ∈ X C: x − x = and adversarially perturbed sample x * = argmax Note that the only difference of adversarial accuracies by maximum perturbation norm and exact perturbation norm is that their allowed regions of adversarially perturbed sample x *, i.e. x: x − x ≤ vs. x: x − x =. The reason why we are separating them will be explained later. Due to the additional requirement x ∈ X C in adversarially perturbed sample x, pseudo adversarial examples will not be considered in genuine adversarial accuracy and thus give more meaningful adversarial accuracy. Depending on X, genuine adversarial accuracies can be undefined. In other word, genuine adversarial accuracies will be undefined when S max = ∅ or S exact = ∅. Definition 5. We define adversarial accuracy functions a: [0, ∞) → for a classifier f. These functions are defined by measuring adversarial accuracies with varying perturbation norms, but genuine adversarial accuracy function uses slightly modified formula. • (Standard) Adversarial accuracy function (by maximum perturbation norm): ] where x * = argmax • (Standard) Adversarial accuracy function (by exact perturbation norm): ] where x * = argmax • Genuine adversarial accuracy function (by exact perturbation norm): previously allowed perturbation region X = x ∈ R d: x − x < where x ∈ X and x * = argmax Likewise, the only difference of adversarial accuracy functions by maximum perturbation norm and exact perturbation norm is that their allowed regions of adversarially perturbed sample x *. Adversarial accuracy function will be also called the change of adversarial accuracy. Genuine adversarial accuracy function will be conventionally also called the change of genuine adversarial accuracy even if it is not strictly correct. We don't define genuine adversarial accuracy function by maximum perturbation norm. One thing to notice in the S exact in the definition of genuine adversarial accrucacy function is that it useX, i.e. the closure of X. The reason we are usingX instead of X will be explained in subsection 2.2. The additional requirement used in genuine adversarial accuracy function was x ∈ X C = R d − X rather than x ∈ X C. It is because we consider the situation where we continuously increase the exact perturbation norm and we want to ignore already considered points for calculation of adversarial accuracy with smaller perturbation norm. This can also be considered as using samples in previously allowed perturbation region X as a new clean input set X = X. Let's think about a toy example (see Figure 5) with predefined (pre-known) classes in order to simplify the analysis. There are only two classes −1 and 1, i.e. Y = {−1, 1}, and, i.e. we assume uniform prior probability. Let's define three classifiers f 1, f 2 and f 3 for this toy example (see Figure 6). When step function step(x) is defined Notice that natural accuracy for all three classifiers is 1. We now explain the change of adversarial accuracy for f 1 (x) by exact perturbation norm (see top right of Figure 7). When 0 < ≤ 1, we can change the predicted class for x ∈ [1, 1 +) by subtracting, and we can't change the predicted class for x / ∈ [1, 1 +), thus standard adversarial accuracy will be 1 − 1 2. When 1 < ≤ 2, there will be same amount of adversarial examples with = 1, thus (standard) adversarial accuracy will be 1 − 1 2 = 1 2. When 2 < ≤ 3, we can still change the predicted class for x ∈ by subtracting. Addition to that we can also change the predicted class for x ∈ [1 −, −1) by adding and (standard) adversarial accuracy will be − where step(x) = 1 for x ≥ 0 and step(x) = −1 for x < 0. Top: Change of (standard) adversarial accuracy for f 1 (x) by maximum perturbation norm (left) and exact perturbation norm x (right) where x = x − x, Middle: Change of adversarial accuracy for f 2 (x) by maximum perturbation norm (left) and exact perturbation norm x (right), Bottom: Change of adversarial accuracy for f 3 (x) by maximum perturbation norm (left) and exact perturbation norm x (right). Observed behaviors of f 2 and f 3 will be same when we compare the adversarial accuracy by maximum perturbation norm, however, observed behaviors of f 2 and f 3 are different when we compare the adversarial accuracy by exact perturbation norm x. When we think about the change of adversarial accuracy for f 2 (x) by exact perturbation norm, by similar analysis, we can check it will be look like middle right graph in Figure 7 when ≤ 5. However, intriguing phenomenon occurs when > 5. When 5 < ≤ 6, x ∈ [1, − 4) cannot change the predicted class as subtracting or adding will in the same class 1, thus adversarial accuracy will be −5 2. If ≥ 6, adversarial accuracy will be The change of adversarial accuracy for f 3 (x) by exact perturbation norm can be understand similarly with f 2 (x). Now, we move on to the explanation for the changes of genuine adversarial accuracy for f 1 (x), f 2 (x) and f 3 (x) (see Figure 8). When 0 < ≤ 1, previously allowed perturbation region X = (−2 −, −1 +) ∪ (1 −, 2 +). When > 1, previously allowed perturbation region X = (−2 −, 2 +). For calculation of genuine adversarial accuracies, we will consider four points, i.e. S exact = {−2 −, −1 +, 1 −, 2 +}, when 0 < ≤ 1 (point 0 will be counted twice when = 1) and two points, i.e. S exact = {−2 −, 2 +}, when > 1. Note that if we did not use closure in the definition of S exact , S exact = {−2 −, 1 −}, when 0 < ≤ 1 and S exact = {−2 −}, when > 1. This will ignore many points and can not measure proper robustness of classifiers. In the change of genuine adversarial accuracy for f 1 (x), when 0 < ≤ 1, −2 −, −1 + and 2 + will be non-adversarial perturbed samples and 1 − will be adversarial example, and thus a gen;exact = 3 4 = 0.75. When > 1, −2 − and 2 + will be non-adversarial perturbed samples, and thus its genuine adversarial accuracy is 1. When considering the change of genuine adversarial accuracy for f 2 (x), for 0 < < 1, −2 −, −1 +, 1 − and 2 + will be non-adversarial perturbed samples, and thus a gen;exact = 1. When = 1, −2 −, 1 − and 2 + will be non-adversarial perturbed samples and −1 + will be adversarial example, and thus a gen;exact = 3 4 = 0.75 (Actually, 1 − = 0 = −1 +, but they counted twice.). When 1 < ≤ 2, −2 − and 2 + will be non-adversarial perturbed samples, and thus a gen;exact = 1. However, when > 2, only 2 + will be non-adversarial perturbed samples and −2 − will be adversarial example, and thus a gen;exact = We introduce Lexicographical (Standard) Robustness (LSR or LR) which is a total preorder based on adversarial accuracy functions by the exact perturbation norm. Furthermore, we explain why LSR is not enough to specify a human-like classifier and why we need Lexicographical Genuine Robustness (LGR). From this, we suggest a candidate oracle classifier what we called "Optimal Lexicographically Genuinely Robust Classifier (OLGRC)". Let's say we have two classifiers f 1 and f 2 for given data D ⊆ X × Y (Here, we are considering general classifiers and not f 1 and f 2 for our toy example.). Let a 1, a 2: [0, ∞) → be the corresponding standard adversarial accuracy functions by exact perturbation norm for f 1 and f 2, respectively. Definition 6. We define a total preorder of classifiers called Lexicographical Standard Robustness (LSR). • We say "f 2 is lexicographically more robust (LR) than • "f 2 is lexicographically equivalently robust (LR) with f 1 " or denote " The reason why we consider adversarial robustness against varying magnitudes of perturbations and not a fixed maximum perturbation norm is that increasing robustness on a fixed maximum perturbation norm will not give a classifier that has human-like robustness as explained in 1.2 (The first property in 1.3.). The defined (total) preorder prioritizes the robustness for smaller perturbation norm because more information in the samples can be lost when larger perturbation is allowed, and thus adversarial accuracy for larger perturbation norm is less important (The third property in 1.3.). This prioritization is also related to the observation that we need to avoid increasing robustness against pseudo adversarial examples who are more likely to occur when the magnitude of the perturbation is large (It is also connected to the second property in 1.3, but in an incomplete way as samples used for adversarial accuracy with small perturbation magnitude can be repeatedly used for larger perturbation magnitudes.). Furthermore, there is also a reason for using adversarial accuracy by exact perturbation norm not by maximum perturbation norm. That was because using adversarial accuracy by exact perturbation norm enables further discretibility as shown in Figure 7. Let's go back to the toy example 2 and three classifiers f 1, f 2 and f 3 for that toy example. According to the Lexicographical Standard Robustness (LSR), we have f 1 < LR f 3 < LR f 2. Then, can we say f 3 is better than f 1, and f 2 is better than f 3? Well, it is true in terms of Standard Robustness only. However, in the following subsection 3.2, we argue why f 2 can be better than f 3 in other aspects. One thing to note here is that if we define f 4 (x) = step(x) − step(x − 4), we can check f 2 = LR f 4 while f 2 = f 4. Hence, LSR doesn't have an antisymmetric property, thus it is not a partial order. In the previous subsection, we explained that the total preorder based on Lexicographical Standard Robustness (LSR) can handle the first and third properties in subsection 1.3, but only incompletely for the second property. To also handle the second property, we use genuine adversarial accuracy function which ignores already considered points for calculation of adversarial accuracy. Let's say we have two classifiers f 1 and f 2 for given data D ⊆ X × Y (Again, we are referring general classifiers and not f 1 and f 2 for our toy example.). Let a 1, a 2: [0, ∞) → be the corresponding genuine adversarial accuracy functions by exact perturbation norm for f 1 and f 2, respectively. • We say "f 2 is lexicographically genuinely more robust (LGR) than • "f 2 is lexicographically genuinely equivalently robust (LGR) with f 1 " or denote " Let's go back again to the toy example 2 and classifiers f 1, f 2 and f 3 for that toy example. According to the Lexicographical Genuine Robustness (LGR), we have Let's consider the perturbations needed to change the predicted classification . Similar to the gradients of differentiable function, the perturbations can be considered as interpretations of classifier as they can change the predicted class. When we think about changing de facto classes, we need positive perturbation x = x − x ∈ in order to change class −1 to class 1, and we need negative perturbation x ∈ (−4, −2) in order to change class 1 to class −1. From this we can see that direction of the perturbations can explain the change of de facto classes. Considering the perturbations needed to change the predicted classes for classifier f 3, we need positive perturbation x ∈ (1, ∞) in order to change class −1 to class 1, and we need negative perturbation x ∈ (−∞, −1) in order to change class 1 to class −1. Note that direction of the perturbations can explain the change of de facto classes. Considering the perturbations needed to change the predicted classes for classifier f 2, we need negative perturbation x ∈ (−6, −1) in order to change class 1 to class −1. However, not only positive perturbation x ∈ (1, ∞) can change class −1 to class 1, but also negative perturbation x ∈ (−∞, −2) can change class −1 to class 1. Hence, the direction of the perturbations no longer explain the change of de facto classes for f 2. We saw that the directions of the perturbations of classifier f 3 explain more the change of de facto classes than classifier f 2. Also, when the Occam's razor principle was considered, we would prefer classifier f 3 over f 2 as they have same standard adversarial robustness for x ≤ 4 and f 2 has one more decision boundary point than f 3, i.e. more complex than f 3. Optimal Lexicographically Genuinely Robust Classifier (OLGRC) is defined as the maximal classifier based on Lexicographical Genuine Robustness (LGR), i.e. this classifier o satisfies either o = LGR g or o > LGR g for any classifier g. OLGRC is determined by expanding explored regions. If each expansion step is (almost everywhere) uniquely determined and expansion can fill the whole space R d, there will be unique OLGRC (in almost everywhere sense). Whether there is unique OLGRC (in almost everywhere sense) or not will be determined by the definition of the metric. We do not cover the detailed conditions for uniqueness. The behavior of OLGRC is similar to the behavior of the support vector machine (SVM) in that its boundary tries to maximize its distance (margin) to the data points. However, linear SVM can only be trained for linearly separable problems even if we assume exclusive belonging settings. On the other hand, Kernel SVM tries to maximize its distance based on the norms of the feature space. Thus, it is probably vulnerable to adversarial attacks in the input set while OLGRC tries to maximize its distance based on the norms of the input set in order to increase adversarial robustness. When we think about the problem setting in the toy example 2, the classifier f 3 is the OLGRC as it's impossible to have a classifier whose change of genuine adversarial accuracy is higher than f 3. We are going to use l 1, l 2, · · · to denote loss functions in this section unlike section 1.1 which were used to represent l p norms. As mentioned in the second properties of the human classification, we need a method that estimates whether a perturbed sample x has de facto class or not to avoid using pseudo adversarial examples in adversarial training. To do that, we train a discriminator that is trained to distinguish clean samples and adversarially perturbed samples. Even if its classification is incomplete because of the overlapping samples, this discriminator allows us to avoid using pseudo adversarial examples for adversarial training. Note that this discriminator has a similar role with the discriminator in Generative Adversarial Nets in that its gradients will be used to generate adversarial examples. In our training method, we will use different magnitudes of perturbations Then, the discriminator will assign corresponding classes for each magnitude. As we need to estimate previously allowed perturbation region X, we provides two different inputs for each class: adversarially perturbed samples L(θ, x, c x) and their opponents x * * = argmin. When we have a discriminator, we will use lexicographical optimization (that will be mentioned in 4.2) to prioritize by avoid generating samples in the previously allowed perturbation region using the discriminator, i.e. to make x * ∈ X C, and to make the perturbed samples adversarial. Gradient Episodic Memory (GEM) was originally developed to prevent catastrophic forgetting which indicates the situation when a network was trained on some tasks, and trained on a new task after finishing the train on the previous tasks, then the network performs poorly (forget to perform well) on the previous tasks. Gradient Episodic Memory (GEM) is a method that enables to minimize the loss for task t without increasing losses for all previous task k < t locally. It is based on first-order approximation of the loss and angles between different loss gradients. To our best knowledge, the lexicographical optimization of neural networks was only used to avoid catastrophic forgetting in continual learning 6. However, we argue that lexicographical optimization of neural networks is not only needed for traditional multi-task learning (MTL), but also for single task learning (STL). Single task learning problems can be described as learning tasks that have only one target loss. However, we often add regularization terms in the training loss in order to prevent over-fitting. As reducing the main loss (target loss) is more important than reducing the regularization terms, we can use lexicographical optimization by prioritizing the main loss. Progressively growing generative adversarial networks uses images with different complexity. We can also think about using lexicographical optimization in their model so that the discriminator and the generator make sure to correctly learn simple structures first. As Lexicographical Genuine Robustness (LGR) also considers multiple accuracies with preference, it can also be considered as a problem that requires lexicographical optimization. To better understand GEM, let us assume that there are losses l 1 (θ), · · ·, l T (θ) with lexicographical preference, in other words, we want to reduce l t (θ) without increasing l 1 (θ), · · ·, l t−1 (θ) for t ∈ {1, · · ·, T}. We also have (pre-projection) parameter updates g 1, · · ·, g T where g t = − ∇l t (θ) and is a learning rate. We locally satisfy lexicographical improvement when g t, g k ≥ 0, ∀k < t. If it is not satisfied, we can project g t to a nearestg t such that it satisfies g t, g k ≥ 0, ∀k < t, i.e.g t = argmin g,g k ≥0,∀k<t g t −g 2. As this problem is a quadratic program (QP) problem, they suggested solving this by its dual problem and recoveringg t when t p (where p is the number of parameters in the neural network). Unlike continual learning that only reduces loss for current task without forgetting previous tasks, we will reduce multiple losses simultaneously with lexicographical preferences. For each lexicographical training step, we can apply weights update for task 1 to task T. But, it requires to calculateg t for each task t and require much computational complexity. In stead of applying several small steps for different tasks, we suggest to apply only one combined weights update for each lexicographical training step. We will call this approach as "Onestep method". When we have suggested parameter updatesg 1, · · ·,g T, let's consider their weighted meang Onestep = T t=1 α tgt where α 1, · · ·, α T ≥ 0 and It means that we can have same lexicographical training effect by simply applying the combined weights updateg Onestep. Considering adversarial robustness for different perturbation itself is not new. As standard accuracy can be considered as adversarial accuracy with zero perturbation, measuring standard accuracy and adversarial accuracy can be regarded as an example. Recently, a research considered the model's robustness against multiple perturbation types and suggested adversarial training schemes for multiple perturbation types. However, their adversarial training methods ("Max" and "Avg" strategies) did not consider the different importance of adversarial accuracy with different magnitudes. Similar concepts with pseudo adversarial examples and problems of using them for adversarial training have been studied. The concept of pseudo adversarial example is similar to the concept called "invariance-based adversarial example" whose predicted class by f is the same with the original class c x even if the predicted class by an oracle classifier o is changed. However, their definition requires an oracle classifier o which is hard to be defined while our definition requires predefined class for samples in clean data set X, and thus it is easier to do theoretical analysis. Invalid adversarial example is also similar to pseudo adversarial example. Their definition assumes data distribution of each class should be a manifold which limits the behavior of data distribution while we don't set manifold requirement in order to consider every possible situation. There were some attempts to balance the accuracy of clean data and accuracy on perturbed samples of classifiers. MixTrain uses adversarial training by dynamically adjusting the hyperparameter α for adversarial training. TRADES (TRadeoff-inspired Adversarial DEfense via Surrogate-loss minimization) method tries to minimize the difference between predicted of adversarially perturbed samples and predicted clean samples instead of minimizing the difference between predicted of adversarially perturbed samples and clean labels. To our best knowledge, there were no attempts to understand the different importance of adversarial accuracies of different magnitudes and prioritized training methods for adversarial robustness. We also handle the problem of simply increasing standard adversarial robustness, i.e. simply finding classifiers who are lexicographically more robust (LSR) than others. In order to compare the different training methods, we experimented with 5 different training methods: standard (non-adversarial) training, standard adversarial training, TRADES, OLSRC and OLGRC. OLSRC refers to the model that trained by applying Onestep method in subsection 4.3 without applying the adversary generation method that avoids generating samples in the previously allowed perturbation region in subsection 4.1. OLGRC refers to the model that trained by applying Onestep method in subsection 4.3 with adversary generation method that avoid generating samples in the previously allowed perturbation region in subsection 4.1. We used PGD method (using exact perturbation norms) to generate adversarially perturbed samples. We used ADAM algorithm to train the discriminator for OLGRC. We found that using mini-batch training for lexicographical optimization might not work well and lexicographical optimization would require full batch training. As lexicographical optimization uses different weights update from a standard method (which is using more than one objective function), simply using mini-batch gradients update can in catastrophic forgetting in other mini-batches. In other words, even if a weights update with lexicographical optimization can improve losses for current mini-batch satisfying the lexicographical improvement, as losses functions on different mini-batch will be different from current mini-batch, the current weight update can increase losses in different mini-batch. In order to avoid this problem, we applied full batch training in all experiments. We did not plot for the changes of genuine adversarial accuracy in our experiments. We think it is unnecessary to plot them for toy example 2. It is impossible to plot the changes of genuine adversarial accuracy for the MNIST experiment as we don't know the actual data distribution and don't have predefined classes for all data. (Note that even if we can use discriminators to estimate them, the discriminators depends on the trained classifiers and estimated changes of genuine adversarial accuracy may not be comparable.) We randomly generated 100 training and 100 test samples from the toy example. Fully connected neural network with one hidden layer (with 256 hidden neurons and leaky-ReLU non-linearity with parameter 0.2) was used for experiments. Full batch training was used with learning rate of 0.015 for 1000 epochs (iterations). Gradient descent algorithm was used for weights update. 6.1.1 The first experiment: examining the effect of lexicographical optimization In order to see the effect of lexicographical optimization in adversarial training, in this experiment, we only used perturbation norm 4 for adversarial attacks. We used α = 0.5 for standard adversarial training. In order to apply comparable effects on the training of OLSRC, we used α 1, α 2 = 0.5 for weights of Onestep method and 10 −10 was used for numerical stability in GEM algorithm. We used 1 λ = 1.0 for TRADES training. Standard (non-adversarial) training and OLGRC were not experimented. Comparing the change of accuracies and losses by iterations in Figure 9, we can observe that training processes of standard adversarial training and TRADES are not stable and classifiers can not be trained properly as both methods don't have prioritization of losses. (It seems TRADES method is less fluctuating than standard adversarial training, but it could be because of different effect of loss function.) On the other hand, training of OLSRC is much more stable as it prioritizes natural cross-entropy loss. Comparing the plots for change of adversarial accuracy in Figure 10, the final classifier obtained by standard adversarial training achieved 0 for both natural accuracy and adversarial accuracy (perturbation norm: 4). The final classifier obtained by TRADES training achieved 1.0 natural accuracy and almost 0 adversarial accuracy (perturbation norm: 4). However, it might achieve 1.0 natural accuracy by chance considering the fluctuating training accuracy. Final OLSRC achieved 1.0 natural accuracy and about 0.5 adversarial accuracy (perturbation norm: 4). In order to see the effect of avoiding already explored regions, in the second experiment, we used exact perturbation norms 1, 2, 3, 4, 5, 6 for adversarial attacks. Only OLSRC and OLGRC were experimented. We used for weights of Onestep method and 10 −3 was used for numerical stability in GEM algorithm. When the generated adversarially perturbed samples using discriminator were observed, we can check that the perturbation process avoids already explored regions even though it is incomplete. For example, when = 4, 5, perturbed samples went to the right direction without making any mistake (Recall that previously allowed perturbation region X is (−2 −, 2 +) when > 1, and in order to avoid already explored points, perturbed samples need to move outward.). Estimated p(x ∈ X C |x) also roughly capture the regions that need to be explored. Comparing the final classifiers and changes of adversarial accuracy in Figure 12, we can observe the shape of the trained OLGRC and its changes of adversarial accuracy are quite similar to f 3 in section 2 which is the theoretical OLGRC. Notice that it was not achievable when we only used training method for OLSRC as shown in the figure. Finding a human-like classifier Figure 11: Plotted graphs show estimated probabilities that the input is not in the previously allowed perturbation region, i.e. estimated p(x ∈ X C |x). Red: [−2, −1) and blue: dashed lines represent the regions for class −1 and class 1. Generated adversarially perturbed samples using discriminator were color plotted class −1: red and class 1: blue. In order to prevent catastrophic forgetting in different mini-batches in mini-batch training, we only used randomly sampled 2000 samples as training data and full batch training was used with learning rate 0.001 for 2000 epochs (iterations). Note that our will not be comparable with other previous analysis on MNIST data because we are using smaller training data. For this experiment, we used common architecture with two convolution layers and two fully connected layers which can be found at https://github.com/MadryLab/mnist_challenge. In order to speed up the training, we applied the ADAM algorithm after projections were applied because of the easiness of implementation. However, as applying adaptive gradient optimization after projections might violate lexicographical improvements, we speculate that it would be better to apply projection after adaptive gradient optimization method was applied. We used the Projected Gradient Descent method with 40 iterations to generates adversarial attacks. Only l 2 norm 4 adversarial attack is used for adversarial training for l 2 norm robust model and only l ∞ norm 0.3 adversarial attack is used for adversarial training for l ∞ norm robust model. We used α = 0.5 for standard adversarial training. In order to apply comparable effects on the training of OLSRC and OLGRC, we used α 1, α 2 = 0.5 for weights of Onestep method. We used 1 λ = 1.0 for TRADES training. Note that due to different formulation of losses training of TRADES will not be directly comparable with other training methods. When we compare the of different training methods (shown in Table 1), we can notice that using OLSRC and OLGRC are better than standard adversarial training Table 2: Results on test data when l 2 norm attacks were used for training and test expectation, trained OLSRC was not lexicographically more robust than trained OLGRC (even on the trained data). It could be the of simultaneously reducing more than one loss and applying the ADAM after projections were applied. (When it comes to natural accuracy for both experiments, TRADES achieved the best . It could be because of different formulation of losses. It also achieved the smallest training loss in both experiments among adversarially trained models. Results on training data were not shown.) In this work, we explained why existing adversarial training methods cannot train a classifier that has human-like robustness. We identified three properties of human-like classification: human-like classification should be robust against varying magnitudes of adversarially perturbed samples and not just on a fixed maximum norm perturbations, when we consider robustness on increasing magnitudes of adversarial perturbations, a human-like classifier should avoid considering already considered points multiple times, and human-like classification need to prioritize the robustness against adversarially perturbed samples with smaller perturbation norm. The suggested properties explain why previous methods for adversarial training and evaluation can be incomplete. For example, the second property explains why commonly used evaluation of adversarial robustness may not fully reveal our intuitive understanding of human-like robustness as standard adversarial accuracies don't avoid pseudo adversarial examples. We defined a candidate oracle classifier called Optimal Lexicographically Genuinely Robust Classifier (OL-GRC). OLGRC is (almost everywhere) uniquely determined when dataset and norm were given. In order to train a OLGRC, we suggested a method to generate adversarially perturbed samples using a discriminator. We proposed to use Gradient Episodic Memory (GEM) for lexicographical optimization and an approach to applying GEM when simultaneously reducing multiple losses with lexicographical preferences. From the first experiment on the toy example from section 2, we showed that lexicographical optimization enables stable training even when other adversarial training methods failed to do so. The second experiment on the same toy example showed that we can use discriminator to roughly generate adversarially perturbed samples by avoiding already explored regions. Because of that, we could train a classifier that is similar to the theoretical OLGRC. From the experiment on the MNIST data, we showed that our methods (OLSRC and OLGRC) achieved better performances on natural accuracy and adversarial accuracy than using standard adversarial training method. In our work, we applied GEM method to adversarial training which is not traditionally a multi-task learning (MTL) problem. This perspective also leads us to use multiobjective optimization (without lexicographical preference) to the problems those were not considered as such. For example, one can use multiobjective optimization to train a single ensemble model that reduces losses in different datasets instead of training different models separately and averaging them. Multiobjective optimization can be used to find an efficient black-box attack by finding adversarial examples that can fool a list of models. By replacing the calculation of an average, it can also be used to smoothen the interpretations of a model. Gradient episodic memory (GEM) with standard gradient descent optimization method is slow and it needs to be combined with adaptive gradient update algorithms. One needs to try applying adaptive gradient update algorithms before the projection was applied. Also, GEM with mini-batch training cannot prevent not increasing losses in the other mini-batches. It is a serious limitation in deep learning applications. Future work needs to find a way to handle this problem. To simplify the problem of finding a human-like classifier, we assumed the exclusive belonging which is unrealistic in many problems. We need analysis when this assumption is violated. We might need to consider easing the lexicographical preference as we expect to get the accuracy that is less than 1 when the exclusive belonging assumption is violated. Another approach would be estimating the hypothetical original data which satisfies the exclusive belonging assumption. In that approach, we consider current data are obtained by adding some input or label noises to the unknown original data. Our training method will find a classifier that is robust against one form (l 1, l 2 or l ∞) of adversarial attacks with different magnitudes. However, we need to find a classifier that is robust against many forms of adversarial attacks (including shift, rotation, spatial transformation, etc.) with different magnitudes as attackers can try different kinds of attacks to exploit the classifier. Our model suggest a (almost everywhere) unique classifier that is robust against one form of adversarial robustness with some conditions. Because of this, in order to find a classifier that is robust against many forms of adversaries, we need to define a combined metric (or its generalization). Figure 13, Below: Change of adversarial accuracy for f (x) by exact perturbation norm x. Notice that f does not satisfy non-increasing Lexicographical Standard Robustness property for 2 < x < 3. We can think of the same toy example in section 2 except for the prior probability. For example, p(c = −1) =. Then, it might be more reasonable to depending on a weighted version of when we define the change of adversarial accuracy functions, i.e. it will be depending on p(c=cx) rather than depending on. 9.3 Interpretation of a classifier: Negative Adversarial Remover (NAR) Definition 8. We define decision boundary (DB) for a classifier f and a class c ∈ Y. • Decision boundary: DB c = x ∈ R d: ∀N (x), ∃x 1, x 2 ∈ N (x) such that f (x 1) = c, f (x 2) = c where N (x) is a neighborhood of x. Note that when f is calculated from an accessible differentiable function g, i.e. f (x) = argmax c∈Y g(x) c, DB c is not equivalent to N B c = x ∈ R d: ∀N (x), ∃x 1, x 2 ∈ N (x) such that g(x 1) c ≥ p(C = c), g(x 2) c < p(C = c) x ∈ R d: g(x) c = p(C = c) when prior is not uniform. N B c will be called neutral boundary (NB). Definition 9. We define negative adversarial remover (NAR) and nearest decision boundary point (NDBP) for a classifier f, a sample x ∈ R d and a class c ∈ Y. • Negative adversarial remover: NAR c (x) = − argmin Note that NAR and NDBP for a sample x can be more than one points. One can check that x = NAR c (x) + NDBP c (x). This indicates that when f (x) = c, NAR c (x) can be an interpretation of the sample x as it is the perturbation that change a point in the decision boundary, i.e. NDBP c (x), to sample x. NDBP c (x) is also similar to the concept called baseline in Integrated Gradients interpretation method while NDBP c (x) is dependent on sample x unlike baseline which will be predefined by users. If f is calculated from an accessible differentiable function g, i.e. f (x) = argmax c∈Y g(x) c, we can use DeepFool algorithm or Fast Adaptive Boundary (FAB)-attack to estimate NAR c (x) when c = f (x). If we only have f, we can use Boundary attack or HopSkipJumpAttack to estimate NAR c (x) when c = f (x).

represent this text to find a much much longer article about it with say hundreds-thousands of words!

We try to design and train a classifier whose adversarial robustness is more resemblance to robustness of human.

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.


Question: Represent the scientific article to find a short TLDR

Question: Generating complex discrete distributions remains as one of the challenging problems in machine learning. Existing techniques for generating complex distributions with high degrees of freedom depend on standard generative models like Generative Adversarial Networks (GAN), Wasserstein GAN, and associated variations. Such models are based on an optimization involving the distance between two continuous distributions. We introduce a Discrete Wasserstein GAN (DWGAN) model which is based on a dual formulation of the Wasserstein distance between two discrete distributions. We derive a novel training algorithm and corresponding network architecture based on the formulation. Experimental are provided for both synthetic discrete data, and real discretized data from MNIST handwritten digits. Generative Adversarial Networks (GAN) BID3 have gained significant attention in the field of machine learning. The goal of GAN models is to learn how to generate data based on a collection of training samples. The GAN provides a unique training procedure by treating the learning optimization as a two player game between a generator network and discriminator network. Since the learning process involves optimization over two different networks simultaneously, the GAN is hard to train, often times unstable BID11. Newly developed models such as the Wasserstein GAN aim to improve the training process by leveraging the Wasserstein distance in optimization, as opposed to the Kullback-Leibler or Jensen-Shannon divergences utilized by the original GAN.A source of interest in generative models arises from natural language processing. In natural language applications, a generative model is necessary to learn complex distributions of text documents. Although both the GAN and Wasserstein GAN approximate a distance between two continuous distributions, and use a continuous sample distance, prior research efforts BID4 BID12 BID10 have applied the models to discrete probability distributions advocating for a few modifications. However, using a continuous sample distance for the discrete case may lead to discrepancies. More precisely, as will be demonstrated via explicit examples, a small continuous distance does not necessarily imply a small discrete distance. This observation has potentially serious ramifications for generating accurate natural language text and sentences using GAN models. To address the above issues, we propose a Discrete Wasserstein GAN (DWGAN) which is directly based on a dual formulation of the Wasserstein distance between two discrete distributions. A principal challenge is to enforce the dual constraints in the corresponding optimization. We derive a novel training algorithm and corresponding network architecture as one possible solution. Generative Adversarial Networks (GANs) BID3 ) model a sample generating distribution by viewing the problem as a two-player game between a generator and a discriminator which is an adversary. The generator takes an input from a random distribution p(z) over a latent variable z, and maps it to the space of data x. The discriminator takes inputs from real data and Table 2: Example of a large gap between discrete and continuous distances for a discrete sample with 9 classes. Training sample: samples from the generator, and attempts to distinguish between the real and generated samples. Formally, the GAN plays the following two player minimax game: DISPLAYFORM0 DISPLAYFORM1 where D is the discriminator network and G is the generator network. In theory, the GAN approximates the Jensen-Shannon divergence (JSD) between the generated and real data distribution. showed that several divergence metrics including the JSD do not always provide usable gradients. Therefore, optimization based on JSD minimization, as incorporated in the GAN, will not converge in certain cases. To overcome the problem, proposed the Wasserstein GAN which is an approximation to the dual problem of the Wasserstein distance. The authors showed that the Wasserstein distance provides sufficient gradients almost everywhere, and is more robust for training purposes. The dual problem of the Wasserstein distance involves an optimization over all 1-Lipschitz functions BID13. The Wasserstein GAN approximates the dual problem by clipping all network weights to ensure that the network represents a k-Lipschitz function for some value of k. A recent variant of the Wasserstein GAN BID4 enforced the k-Lipschitz property by adding a gradient penalty to the optimization. Although the formulation of the Wasserstein GAN approximates the Wasserstein distance between two continuous distributions, using a continuous sample distance x − y, existing research efforts BID4 BID12 BID10 have directly used it to model discrete probability distributions by adding the following modifications. Each component of the input vectors of training data is encoded in a one-hot representation. A softmax nonlinearity is applied in the last layer of the output of the generator to produce a probability that corresponds with the one-hot representation of the training data. During training, the output of the softmax layers becomes the input to the critic network without any rounding step. To generate a new sample, an argmax operation over each generator's softmax output vectors is applied to produce a valid discrete sample. The usage of continuous sample distance in the standard Wasserstein GAN for discrete problems as described above creates some discrepancies in the model. These discrepancies are illustrated in TAB0. In TAB0, we have two different outputs from the generator's softmax with the same real sample reference. Although the first softmax output produces the same value as the real sample when it is rounded using argmax (hence has discrete distance 0 to the real sample), it has a larger continuous distance compared to the second softmax output which produces one mistake when rounded (has discrete distance 1 to the real sample). In the discrete case, with a large number of classes, as shown in Table 2, even though the generator output produces a discrete sample with the same value as the real sample when rounded, there still exists a very large continuous distance. This difference between continuous and discrete distance becomes greater for a larger number of discrete classes. Motivated to correct modeling discrepancies as described in Section 2, which occur due to the mismatched use of the standard Wasserstein GAN in discrete problems, we propose a new GAN architecture that is directly based on the Wasserstein distance between two discrete distributions. Let a vector x = (x, x,...) be a discrete multivariate random variable where each component x(i) can take discrete values from {1, 2, 3, . . ., k}. Let P r and P s be two probability distributions over the set of values for x. The Wasserstein distance between two probability distributions P r and P s is defined as: DISPLAYFORM0 The notation Π(P r, P s) denotes the set of all joint probability distributions γ(x, x) whose marginals are P r and P s respectively, and d(x i, x j) denotes the elementary distance between two samples x i and x j. We are particularly interested with the sample distance that is defined as the hamming distance (the sum of zero-one distance of each component), i.e: TAB1 shows an example of the sample distance metric. DISPLAYFORM1 Visible in the formulation above, computing the Wasserstein distance between two discrete probability distributions is a Linear Program (LP) problem for which the runtime is polynomial with respect to the size of problem. However, for generating real-world discrete distributions, the size of problem grows exponentially. For example, if the number of variables in vector x is 100, and each variable can take values in the set {1, 2, . . ., 10} so that k = 10, the size of the LP problem is O(10 100) reflecting the number of configurations for x. The ing LP is intractable to solve. We follow a similar approach as in by considering the dual formulation of Wasserstein distance. Kantorovich duality BID2 BID13 tells us that the dual linear program of the Wasserstein distance can computed as: max DISPLAYFORM2 DISPLAYFORM3 The function f maps a sample to a real value. Note that unlike for the continuous Wasserstein distance, in which the maximization is over all 1-Lipschitz functions without additional constraints, the maximization above is over all functions that satisfy the inequality constraints in Eq. 5. The dual formulation of the Wasserstein distance is still intractable since the maximization is over all functions that satisfy the inequality constraints. We aim to approximate the dual Wasserstein distance formulation by replacing f with a family of parameterized functions f w that satisfy the inequality constraints. The parameterized functions f w are modeled using a neural network. Unfortunately, it is difficult to construct a neural network architecture to model f w while also explicitly satisfying the inequality constraints involving the discrete sample distance defined in Eq. 3.To overcome the problem of approximating f with neural networks, we note that the maximization in the dual formulation is equivalent to the following optimization: DISPLAYFORM0 subject to: DISPLAYFORM1 where DISPLAYFORM2 Instead of approximating f (x), we aim to design a neural network architecture that approximates h(x, x) and satisfies the inequality constraints in Eq. 5. The key idea is that this new optimization is equivalent to the original dual formulation of the Wasserstein distance (explained in the sequel), even though the optimal form for h is not explicitly specified. Our selected architecture for the generator network employs the same softmax nonlinearity trick for the standard Wasserstein GAN described in Section 2. The generator network is a parameterized function g θ that maps random noise z to a sample in one-hot representation. The last layer of the generator network utilizes softmax nonlinearity to produce a probability which corresponds with the one-hot representation of the real sample. Our key modeling difference lies in the critic network. The critic network takes two inputs, one from the real samples, and one from the output of the generator. The architecture of the critic network is visualized in FIG2.Let y ∈ m×k be the one-hot representation of x where m is the number of variables and k is the number of classes for each variable. The critic network takes two inputs: y from the real training data, and y from the output of the generator network. Let us define ρ w as a parameterized function that takes input (y, y) ∈ 2×m×k and produces an output vector v ∈ [−1, 1] m. From the generator output y, we compute the rounded samplex. Let u ∈ {0, 1} m be a vector that contains the element-wise zero one distance between a real training sample x and rounded samplex from the generator, i.e. u(i) = I(x(i) =x (i)). We define our approximation to the function h as a parameterized function h w that is defined as h w = u T v = u T ρ w (y, y). The "filter" vector u ensures that the output of h w always satisfies the inequality constraints DISPLAYFORM3 5. An illustration of this neural network architecture and construction is provided in FIG2.As we can see from FIG2, the critic network consists of two separate sub-networks. The first sub-network takes input from a batch of samples of the training data, while the second sub-network takes input from a batch of samples produced by the generator. Each sub-network has its own set of intermediate layers. The outputs of the first and second layers are concatenated and taken as an input to a fully connected layer which produces a tensor of size n × m. The dimension n indicates the number of samples in a batch, and m is the number of variables. To produce a tensor v whose values range from -1 to 1, a tanh nonlinearity is applied. The "filter" tensor u is applied to v via an element-wise multiplication. The output of the critic network is calculated by taking the sum of the of the element-wise multiplication of u and v, yielding a vector of n elements containing the value of h w (y, y) for each pair of real and generated samples. We also included additional modifications based on theory to facilitate the training of networks. Note that since h(x, x) = f (x) − f (x), we can attempt to enforce this optimum condition known from theory. If we flip the inputs to h w we will get the negative of the output; i.e. h w (y, y) = −h w (y, y). To model this fact, we randomly swapped the sample from the real training data and generator output so that some of the real data was fed to the first sub-network and some to the second sub-network. If a pair of samples was flipped, we multiplied the output of the network with −1. Another modification that we applied to the network was to introduce a scaling factor to the softmax function such that the output of the scaled softmax was closer to zero or one. Specifically, we applied the function: softmax(x)(i) = exp(k·x(i)) j exp(k·x(j)), for some constant k ≥ 1. The training algorithm for our proposed architecture is described in Algorithm 1. Algorithm 1 Discrete Wasserstein GAN 1: Input: learning rate α, batch size n, the number of critic iteration per generator iteration n critic 2: repeat 3: DISPLAYFORM4 Sample a batch from real data DISPLAYFORM5 Sample a batch of random noise DISPLAYFORM6 end for 8:Sample a batch from real data DISPLAYFORM7 Sample a batch of random noise DISPLAYFORM8 10: DISPLAYFORM9 In contrast with the continuous GANs where many models have been proposed to improve the performance of GAN training, only a few GAN formulations have been proposed for modeling discrete probability distributions. BID4 use the standard continuous Wassersten GAN with adjustments described in Section 2. Similar techniques are used by BID12 to address several natural language generation tasks. augment the original GAN architecture with a maximum likelihood technique and combine the discriminator output with importance sampling from the maximum likelihood training. propose a Boundaryseeking GAN (BGAN) that trains the generator to produce samples that lie in the decision boundary of the discriminator. BGAN can be applied for discrete cases provided that the generator outputs a parametric conditional distribution. Other GAN models BID15 exploit the REINFORCE policy gradient algorithm BID14 to overcome the difficulty of backpropagation in the discrete setting. BID6 combine adversarial training with Variational Autoencoders BID7 to model discrete probability distributions. Evaluating the performance of generative models objectively and effectively is hard, since it is difficult to automatically tell whether a generated sample is a valid sample from the real distribution. Previous research advocates user studies with human graders, especially in image generation tasks, or proxy measures like perplexity and corpus-level BLEU in natural language generation. However, such techniques are far from ideal to objectively evaluate the performance of GAN models. To address the limitations above, we propose a synthetic experiment that captures the complexity of modeling discrete distributions, but still has a simple strategy to objectively evaluate performance. The synthetic experiment is based on a classic tic-tac-toe game. We generalize the classic 2 player tic-tac-toe game to include arbitrary k players and arbitrary m-by-m board sizes (rather than the default 3-by-3 board). The goal is to model the true generating distribution P r which is the uniform distribution over valid configurations of the board when a generalized tic-tac-toe game has ended (e.g. the final game state). We generalized the concept of a valid board in 3-by-3 games, in which one player has a winning state and marks filling a full column, row, or diagonal. For the purpose of our experiment, we made a simplification to the valid rule, i.e. as long as the board has at least one full column, row and diagonal taken by at least one player, it is considered to be a valid configuration. FIG3 shows examples of valid and non-valid board configurations. In our construction above, it is easy to check if a generated sample is a valid sample under the real distribution. Hence it is possible to validate objectively the performance of a generative model. Furthermore, it is also easy to sample from the real distribution to create synthetic training data. We uniformly sample random board configurations, accepting a sample if it is valid, and rejecting it if invalid. We construct several metrics to track the performance of the model. The first measure is the percentage of valid samples which characterizes the quality of the samples generated by the generator network. For a bigger board the percentage of valid samples does not tell much about the progress of learning since it takes a while to get to a valid sample. We construct another metric which is the average of maximum player's gain. The maximum player's gain for a board configuration is defined as the maximum number of cells taken by a player in a full column, row, or diagonal. FIG4 shows the value of maximum player's gain for three different 5-by-5 board configurations. In the left board, player 2 and 4 have the maximum (3 cells); in the middle board player 2 takes 4 cells; and in the right board, player 2 achieves the maximum of 5 cells. Note that for k-by-k boards, if the average of maximum player's gain is equal to k, it means that all the samples are valid. Therefore, closer average of maximum player's gain to k indicates a better quality of samples. Besides those two metrics, we also track the percentage of unique samples and the percentage of new samples, i.e. samples that do not appear in the training data. In the experiment, we compare our Discrete Wasserstein GAN model with the standard Wasserstein GAN model (with tricks described in Section 2) on 3-by-3 and 5-by-5 board with 2 players and 8 players. Note that the number of classes is equal to the number of players plus one since we need an additional class for encoding empty cells. We restrict the generator and critic networks in both models to have a single hidden layer within fully connected networks to ease training. As we can see from Figure 4, our DWGAN networks achieve good performance (in terms of the average of the percentage of valid samples and the maximum player's gain metrics) much faster than the standard WGAN with softmax and one-hot representation tricks. In both 3-by-3 boards with 2 players and 5-by-5 boards with 8 players our DWGAN networks only take less than a third of the iterations taken by the standard WGAN to achieve similar performance. We observe that our DWGAN networks have a mode collapse problem that occurs after achieving top performances. Figure 5a shows that the DWGAN can achieve the average of maximum player's gain close to 5 for a 5-by-5 board in 500 iterations while maintaining the percentage of unique samples close to 100%. After it produces those diverse samples, the network model begins to suffer from a mode collapse and the percentage of unique samples decrease to less than 10% after iteration 550. Based on our analysis, this behavior is caused by the fact that the network optimizes the function difference DISPLAYFORM0, which tends to cause an advantage if the values of g θ (z i) are not diverse. To overcome this issue, we add a norm penalty to the critic network optimization, i.e: DISPLAYFORM1 where λ is the penalty constant. Figure 5b shows the effect of the norm penalty to the performance of DWGAN and its sample diversity. We observe that the DWGAN network with a norm penalty can achieve 96% valid samples while maintaining the diversity in samples it generates (around 50% unique samples). To model more complex discrete distributions, we used MNIST digits discretized to binary values BID8 as the training data with the goal to generate new digits with our proposed Discrete Wasserstein GAN. As a baseline, we trained a standard Wasserstein GAN on the continuous digit dataset. Similar to our synthetic experiments, we restricted the generator and critic networks to have only a single hidden layer within fully connected networks. Figure 6 shows that our model produces a similar quality of discretized digit images compared to the continuous value digits produced by the standard Wasserstein GAN trained on continuous-valued data. We further generated 100 samples from our DWGAN model, prior to mode collapse, illustrating the diversity of samples. We proposed the Discrete Wasserstein GAN (DWGAN) which approximates the Wasserstein distance between two discrete distributions. We derived a novel training algorithm and corresponding network architecture for a dual formulation to the problem, and presented promising experimental . Our future work focuses on exploring techniques to improve the stability of the training process, and applying our model to other datasets such as for natural language processing. A linear program (LP) is a convex optimization problem in which the objective and constraint functions are linear. Consider a vector variable x ∈ R n, matrix A ∈ R n×m, and vectors c ∈ R n, and b ∈ R m. An LP is given in the following standard form, DISPLAYFORM0 The Lagrange dual function is given by, DISPLAYFORM1 The Lagrange dual problem is to maximize g(λ, ν) subject to λ 0. Equivalently, the dual problem may be written as an LP in inequality form with vector variable ν ∈ R m, DISPLAYFORM2 The dual of the above problem is equivalent to the original LP in standard form. Due to the weaker form of Slater's condition, strong duality holds for any LP in standard or inequality form provided that the primal problem is feasible. Similarly, strong duality holds for LPs if the dual is feasible. Consider discrete probability distributions over a finite set X with cardinality |X |. Assume an elementary sample distance d X (x 1, x 2) for x 1, x 2 ∈ X. The sample distance evaluates the semantic similarity between members of set X. Define P r (x) and P s (x) for x ∈ X as two discrete probability distributions. In this case, we may define the exact discrete Wasserstein distance between P r and P s as a linear program as follows, with D ∈ R |X |×|X | + whose matrix entries correspond to the sample distance DISPLAYFORM0 The dual LP is given as follows. DISPLAYFORM1 At the optimum it is known that ν = −µ, and the dual LP is equivalent to the following optimization problem. Note that there still exist |X | × |X | constraints. DISPLAYFORM2 Example 1. The following example provides a closer look at the dual optimization problem. Consider a finite set X = {1, 2, 3}. Let P s (x) be given by the discrete distribution P s = 0.2, P s = 0.7 and P s = 0.1. Similarly, let P r (x) be given by the discrete distribution P r = 0.4, P r = 0.4, P r = 0.2. Define the elementary sample distance d X (x 1, x 2) = 1 if x 1 = x 2 and d X (x 1, x 2) = 0 if x 1 = x 2. Therefore, the sample distance matrix D for this discrete example is the following: DISPLAYFORM3 The optimal value of the matrix T provides the optimal transport of mass from P s to P r, The objective value of the primal and dual is equal to 0.3 which is the total mass moved from P s to P r. In the solution to the dual problem, ν = [0 −1 0] T and µ = −ν. In this example, it is seen that the optimal ν = −µ. DISPLAYFORM4 For the synthetic experiments, we use Julia v0.5.2 programming language with Knet deep learning framework. Below is the code containing functions needed to generate tic-tac-toe board. # np = number of player for j = 1:c_iter # fake samples z = KnetArray(randn(Float32, nz, n)) fv = netG(wG, z, np, softmax_scaling) # real + fake outputC = netC(wC, rv, fv, np, lambda) tl.log_value("output_C", outputC, itC) tl.log_value("output", outputC, it) gC = netC_grad(wC, rv, fv, np, lambda) tl.log_value("grad_C_mean", mean(map(x -> mean(Array(x)), gC)), itC) tl.log_value("grad_C_std", mean(map(x -> std(Array(x)), gC)), itC) for i in 1:length(wC) wC[i] += lrC * gC[i] end it += 1 itC += 1 end if itC % decayitC == 0 lrC = decayC * lrC end # train generator for j = 1:g_iter z = KnetArray(randn(Float32, nz, n)) outputG = netGC (wG, wC, z, rv, np, softmax_scaling, lambda) tl.log_value("output_G", outputG, itG) tl.log_value("output", outputG, it) gG = netGC_grad(wG, wC, z, rv, np, softmax_scaling, lambda) tl.log_value("grad_G_mean", mean(map(x -> mean(Array(x)), gG)), itG) tl.log_value("grad_G_std", mean(map(x -> std(Array(x)), gG)), itG) for i in 1:length ( DISPLAYFORM0

Represent the short summary for finding the corresponding scientific article (long)

We propose a Discrete Wasserstein GAN (DWGAN) model which is based on a dual formulation of the Wasserstein distance between two discrete distributions.

Represent the TLDR for finding the corresponding scientific article!

This paper 1) characterizes functions representable by ReLU DNNs, 2) formally studies the benefit of depth in such architectures,  3) gives an algorithm to implement empirical risk minimization to global optimality for two layer ReLU nets.