Question:
represent the text to find a very short summary (1-2 sentences max)!
The query could be 'We present a novel approach to train a natural media painting using reinforcement learning. Given a reference image, our formulation is based on stroke-based rendering that imitates human drawing and can be learned from scratch without supervision. Our painting agent computes a sequence of actions that represent the primitive painting strokes. In order to ensure that the generated policy is predictable and controllable, we use a constrained learning method and train the painting agent using the environment model and follows the commands encoded in an observation. We have applied our approach on many benchmarks and our demonstrate that our constrained agent can handle different painting media and different constraints in the action space to collaborate with humans or other agents. Throughout human history, painting has been an essential element of artistic creation. There are many diverse and complex artistic domains with various styles such as watercolor, oil painting, sketching, and so on. As image processing and computer graphics have advanced, there has been a considerable effort to simulate these styles using non-photorealistic rendering (NPR) techniques .; generate compelling using stroke-based rendering. However, most prior methods in NPR are engineered for a specific application or task, and cannot easily adapt to new styles or medium. Recent developments in machine learning have ed in significant advancements in computer vision and computer graphics, including computer-based painting systems. Many visual generative methods based on generative adversarial networks as;;;; have demonstrated promising . Many of these machine learning methods have also been applied to stroke-based rendering tasks, including modeling the brush , generating brushstroke paintings in an artist's style , reconstructing drawings for specific paintings styles , and constructing stroke-based drawings (Ha & Eck (2017a);;; Jia et al. (2019a) ). In this paper, we focus on a more general and challenging problem of training a natural media painting agent for interactive applications. Given a reference image, our goal is to develop a stroke-based rendering approach that can imitate the human drawing or strokes used in generating the image. A key challenge is to develop a method that can learn from scratch without any supervision. In this regard, we present a technique that can handle all inputs and train an agent to manipulate natural painting media such as charcoal, pencil, watercolor, and so on. We build a model-based natural media environment using deep CNN and train a natural media painting agent using model-based reinforcement learning. In order to introduce controls to the agents for interactive applications, we use a constraint representation along with a different framework for training and use the constrained painting agent. These constraints enable the agent to interact with a human or other agents and generate various styles without retraining the model. The novel contributions of our work include: • A method to train an agent that produces a stream of actions subject to constraint for each action. These constraints can include restricting the start location, stroke width, color, and other stroke parameters. • A method to roll out constrained agents so the user can produce new stylistic effects interactively or automatically, as the agent is painting by modulating the action stream. • By incorporate coarse-to-fine strategy, our painting agents can generate high-resolution stylized images using various constraints and paintbrush configurations. We evaluate our algorithm on different paintbrush configurations and datasets to highlights its benefits over prior reinforcement learning based methods. We also employ differing constraint settings to validate our constrained agents and produce new stylistic effects with a single trained model. In this paper, we focus on the stroke-based rendering problem, which renders the reference image with brush strokes. In contrast to image analogy approaches (; ; ; ; ; ; Li et al. (2017;), stroke-based approaches can generate intermediate painting states for interactive purposes. They can also be deployed in various painting environment to generate different artistic effects. One category of approaches for stroke-based rendering uses a heuristics-based method to guide the agent while painting. Notable examples include the method in , which can produce stylized illustrations, and the method in , which can reproduce a colorful painting in different styles. However, it is difficult to extend these methods to different styles of painting because of the hand-engineered features. Another category of approaches uses machine learning techniques to learn the policy. Compared with predefined policies, these methods use machine learning techniques to learn the painting policy, which enables the agent to generate a more natural . Ha & Eck (2017b) train an RNN to learn the latent space of the sketch data and generate the paintings in this latent space, which requires the paired dataset for training. Other approaches use deep reinforcement learning to learn the policy without supervised data. There have been a few attempts to tackle related problems in this domain. Xie et al. (2012; 2013) propose a series of works to simulate strokes using reinforcement learning and inverse reinforcement learning. These prior approaches learn a policy either from reward functions or expert demonstrations. Comparing with these approaches, we use a more general setup which does not rely on rewards engineering. is based on the Deep Q network. trains the discriminator and the reinforcement learning framework at the same time. However, both methods can work on either a small action space or a small observation space. Jia et al. (2019b; a) use proximal policy optimization with curriculum learning and self-supervision to gradually increase the sampled action space and the frequency of positive rewards. train a differentiable painting environment model, which helps learn the painting policy with gradient-based methods. train a model-based RL using differentiable environment and DDPG . This approach is limited by the OpenCV based renderer and the uncontrollable agent. An overview of our approach is given in Fig. 1 This includes the training phase and extracting the underlying neural network as the painting agent is used for the roll-out process. We highlight all the symbols used in the paper in Table 1. In section 4, we present the details of natural media painting agent with a environment model for the agent to interact. Our approach is based on the training scheme described in and uses Deep Deterministic Policy Gradient (DDPG) to train the modelbased painting agent. For the roll-out algorithm, we apply the painting agent directly to the real environment R r as shown in Figure 1. In section 5, we present our representation of the underlying constraints and the underlying techniques to encode and decode these constraints using reinforcement learning. We use the unconstrained agent to identify the constraints, and encode them as observations to train the constrained painting agent, as shown in Figure 3. In order roll out the We use a actor-critic-based reinforcement learning framework to train the painting agent. For each step, the current states of the canvas and the reference image form the observation for the policy network. Based on the observation, the painting agent predicts an action for the neural renderer R n to execute and updates the canvas accordingly. Roll-out of the Natural Media Painting Agent (right): We extract the actor neural network as the painting agent for the roll-out process. We replace the neural renderer R n with the real renderer R r to get the precise synthetic . In this section, we present our algorithm for training a natural media environment model and training a painting agent to interact with this environment model. The renderer of our painting agent generates the corresponding canvas by the given action, which implements the blending functions and other synthetic programs. Unlike the previous learning-based painting approaches Jia et al. (2019a);; , we use a natural media painting renderer MyPaint (libmypaint contributors) for our experiment. Compared with self-defined painting simulators, it provides rich visual effects and sophisticated blending functions. use the same environment but have not explored the various configurations of the paintbrush. Using a pre-existing environment setup helps our approach concentrate on the learning algorithm to make it easy to generalize. The action of our painting agent consists of the configurations it use to interact with the environment. For the action space, we follow the stroke definition of using a quadratic Bezier curve (QBC) of three points ((x 0, y 0), (x 1, y 1), (x 2, y 2)). Each stroke is a 3-point curve. The pressure which affects the blending function is linear interpolated by the value of the start position (x 0, y 0) and the end position (x 2, y 2). We use one color (R, G, B) for each stroke, and the transparency of the pixels within the stroke is affected by the pressure (p 0, p 1). We use one brush size r for each stroke, and the actual stroke width within the stroke can also be affected by the pressure (p 0, p 1). Formally, we represent the action a t as: In practice, a t ∈ R 12. Each value is normalized to. The action is in a continuous space, which makes it possible to control the agent precisely. The observation of our painting agent consists of information to make decisions. In our setup, the observation is represented by the reference image s *, the current canvas s t at step t, and the constraint c t. Formally, o t = (s *, s t). For the constrained agent described in section 5, o t = (s *, s t, c t). c t is a vector drawn from the sub-space of the action space. To unify the observation representation for implementation, we upsample the constraint vector as a bitmap with size of s t to feed into the CNN of the policy network and the value network. Reward is a metric that enables our painting agent to measure effectiveness of the action. Aside from the reinforcement learning algorithm, the definition of the reward can be seen as a guidance for the policy π. For our problem, the goal of the painting agent is to reproduce the reference image s *. First, we define the loss function L between the current canvas s t at step t and the reference image s *. In practice, we use l 2 loss (Jia et al. (2019a) ) and WGAN loss ) for implementation. Then, we use the difference of the losses of two continuous steps as the reward r t. We normalize r t using Eq.2, such that r t ∈ (−∞, 1]. where L is a loss function defined as l 2 or WGAN. For our reinforcement learning setting, the objective is to maximize the discounted accumulative rewards q t = tmax t=1 r t γ t, where the discount factor γ ∈ For this painting environment, we use the rendering function R to represent the transition function. The action only modifies the current painting state s t with the current action a t as s t+1 = R(a t, s t). Inspired by and , we model the behaviors of the real environment renderer R r using a neural network as R n. We use a 4-layer fully connected neural network followed by 3 convolutional neural networks.) with different paintbrush configurations. The top row is rendered by the real renderer R r and the bottom row is rendered by the neural renderer R n. There are two main benefits of modeling the environment. First, the reinforcement learning algorithm may have millions of steps, and it can be computationally expensive to use the real environment because of sophisticated synthetic algorithms. If we simplify the environment model, it can save training time. Second, modeling the transition function of states using neural network makes the entire framework differentiable, and the loss can be backpropagated through the renderer network R n. In practice, we model R r by building a dataset consisting of paired data (a t, s t, s t+1). To simplify the problem, we always start from a blank canvas s 0 so that the R n only need to predict the stroke image from a single action. At run-time, we synthesize the blending function by making the stroke image transparent and adding it to the current canvas s t. The paired dataset {a (i), s (i) } can be collected by randomly sampling the a i and synthesising the corresponding s i. We treat different paintbrushes of MyPaint (libmypaint contributors) as separate models. It is worth noting that we use the environment model R n for the training process, but use the real environment R r for the roll-out process. After we train the natural media painting agent, we make the agent controllable to handle interactive scenarios and be used for generating stylized images. To achieve this goal, one straightforward way is to constrain the agent with a modified action space. However, this method is difficult to generalize to different constraints. For each different constraint c in the same subspace C in action space A, the policy needs to be retrained. For example, if we train a painting agent and constrain its stroke to width 0.8, we still need to train it again when we need an agent to output a stroke with width 0.9. In the rest of this section, we propose a constrained painting agent that, can follow any constraint c drawn from the subspace of action space A. First, we define the constraint representation, including the definition of the constraint, the encoding function, and the decoding function. Then we introduce the training and roll-out schemes of the constrained agent. The constraint is the vector c in the constraint space C, which is the sub-space of the action space A. While fixing the vector c, the agent has to explore the subtraction of A and C as A by sampling a from A and concatenating with c: We can define C by selecting and combining dimensions in A. For example, we can constrain the color of the stroke by defining C as the color space (R, G, B). We can constrain the start position of the stroke by defining C as (x 0, x 1). We can constrain the stroke width of the stroke by defining C as (r). Moreover, we can use the combination of the constraints to achieve complex effects. As shown in Figure 3, the unconstrained agent takes the reference image and the current canvas (s *, s t) as observation while the constrained agent takes the reference (s *, s t, c t) as an observation, which has an additional constraint c t at step t. c t is a vector drawn from the sub-space of the action space. To unify the observation representation for implementation, we encode the constraint vector c t as a bitmap to feed into the CNN of the policy network and the value network. To encode the constraint c t into observation, we upsample the constraint c t as a matrix c t which has the same size of s * and s t, to stack them and feed into our policy network and our value network. To decode the constraint c t into c t, we implement a downsample module within the policy network. The policy network pi can be seen as separate policy: π c and π a; π c outputs the downsampled constraint c and π a outputs the constrained action a. Then we concatenate a and c to form action a. Formally, we have c t = π c (c t), a t = π a (s *, s t, c t), and a t = a t ⊕ c t. After we introduce the constraint c t as part of the observation of the painting agent, we propose the corresponding training scheme. Because the constraint representation is designed for interactive purposes, we need to use either human experts or another agent to train the constrained agent. As shown in Figure 3, we use an unconstrained agent to generate constraints by cascading them together. For each step t, the unconstrained agent takes the reference image and the current canvas (s *, s t) as observations, and outputs the action a in action space A. Then we identify a subspace in A as constraint space C and transfer a t to c t, followed by upsampling the c t as c t, as defined in section 4. After that, the constrained agent takes the additional constraint c t as an observation (s *, s t, c t) and outputs action a t concatenated by c t and a. Figure 3: Framework for Training (Roll-out) the Constrained Agent We train the constrained agent by cascading an unconstrained agent. First, we identify a subspace of the action space and extract the constraint c from the action a. Then we upsample the constraint c and pass it on as an observation of the constrained agent. We use R n as renderer. For roll-out process (dashed line), we extract the constraint from either the user defined constraint or the action of the previous step. We use real renderer R r to get the best visual quality. For the roll-out of the constrained painting agent, we replace the unconstrained agent from the training scheme in Figure 3 with the constraint extracted from either human command or a painting agent's output for each step t. When the constraint is extracted from a human command, the agent can be ordered to place a paint stroke with a given position (x 0, y 0), color (R, G, B), and stroke width r. When the constraint is extracted from painting agent's output, it can be seen that we split the one agent that explores in entire action space A into two agents that explore separate action spaces A 0 and A 1, where A = A 0 + A 1. For training the natural media painting environment, we treat different paintbrushes as different models. We run 50,000 iterations for each model and record the l 2 loss on the validation dataset as shown in Table 2. The learning curves of the training processes are shown in Figure 4 (left). We train the unconstrained painting agent using the fixed envionment model with various dataset. We use hand-written digits images , character images (, face images (, object images as train painting agents. We run 10,000 episodes for each training task. Because of the difference among the dataset, we use 5 strokes to reproduce hand-written digits images, 20 strokes to reproduce character images, 100 strokes to reproduce face and object images. We measure the l 2 loss between the reference images and reproduced images throughout the training process shown as Figure 4 (middle). The reproduced is shown as Figure 5, and the corresponding l 2 loss is shown as Table 3 (left). Figure 5: We trained our natural media painting agents using MNIST, KanjiVG, CelebA, and ImageNet as reference images (left). We generate (right) using R r for the training and validating process. For the roll-out process, we employ a coarse-to-fine strategy to increase the resolution of the shown as Figure 6. First, we roll out the agent with a low-resolution reference image s * and get the reproductionŝ *. Then we divide s * andŝ * into patches and feed the agent as initial observation. The roll-out using various natural media are shown as Figure 7. We train the constrained painting agents using the learning parameters as unconstrained painting agents. We compute the l 2 distance between the reference images and reproduce for the training process with various constraint configurations. To control variates, we use same neural environment R n (charcoal) and dataset (CelebA) for these experiments. We use the color (R, G, B), the stroke width r, the start position (x 0, y 0) and all of them (x 0, y 0, r, R, G, B) as constraints. The learning curves of these constraints are shown as Figure 4 (right) and the l2 loss is shown as Table 3(right). We demonstrate the constrained painting agents as Figure 2, which uses pencil as paintbrush and incorporates coarse-to-fine strategy by dividing the reference images as 4×4 patches. To constrain the color of strokes, we build a color palette by clustering the colors of the reference image. For each action a, we constrain it from a color randomly selected from the color palette. To constrain the stroke width, we use a constant stroke width for the roll-out process. Figure 6: Coarse-to-fine Roll-out We roll out the trained agent using a coarse-to-fine strategy. We first roll out the model treating the reference image and canvas as one patch. Then we divide the reference image and updated canvas into patches and feed to the agent. In this figure, we roll out 100 strokes in each patch. In this paper, we train natural media painting agents that can generate artistic paintings using various natural media, and collaborate with humans and other agents to get different visual effects. We build a model of natural media environment using deep CNN and train a natural media painting agent using model-based reinforcement learning. To introduce controls to the agents for interactive purposes, we propose constraint representation, a framework for training a constrained painting agent, and various roll-out schemes to apply the agent. We demonstrate our algorithm by applying the trained model using various paintbrushes from MyPaint and constraints set up. The experimental show that our algorithm can reproduce reference images in multiple artistic styles. For future work, we aim to extend the proposed algorithm by building a unified model for differing paintbrush configuration. In addition, we will train a hierarchical agent that uses a constrained agent as the low-level policy. We would like to apply our approach on other reference images and use for interactive painting systems. A APPENDIX Figure 9: Roll-out using Various PaintbrushesWe roll out our natural media painting agents trained with various brushes in MyPaint. To increase the resolutions of the generated images, we incorporate the coarse-to-fine strategy. We use 8 × 8 patches for first row and 4 × 4 for second row. Figure 10: Reproduction of Starry Night using Charcoal We roll out our natural media painting agent trained with charcoal brush in MyPaint to reproduce Van Gogh's starry night. We incorporate the coarse-to-fine strategy by dividing the reference image and canvas into 16 × 16 patches. Figure 11: Reproduction of Starry Night using Watercolor We roll out our natural media painting agent trained with watercolor brush in MyPaint to reproduce Van Gogh's starry night. We incorporate the coarse-to-fine strategy by dividing the reference image and canvas into 16 × 16 patches.' and should be close to 'We train a natural media painting agent using environment model. Based on our painting agent, we present a novel approach to train a constrained painting agent that follows the command encoded in the observation.' but very far from 'We use reinforcement learning to train an agent to solve a set of visual arithmetic tasks using provided pre-trained perceptual modules and transformations of internal representations created by those modules.'

Question: A wide range of defenses have been proposed to harden neural networks against adversarial attacks. However, a pattern has emerged in which the majority of adversarial defenses are quickly broken by new attacks. Given the lack of success at generating robust defenses, we are led to ask a fundamental question: Are adversarial attacks inevitable? This paper analyzes adversarial examples from a theoretical perspective, and identifies fundamental bounds on the susceptibility of a classifier to adversarial attacks. We show that, for certain classes of problems, adversarial examples are inescapable. Using experiments, we explore the implications of theoretical guarantees for real-world problems and discuss how factors such as dimensionality and image complexity limit a classifier's robustness against adversarial examples. A number of adversarial attacks on neural networks have been recently proposed. To counter these attacks, a number of authors have proposed a range of defenses. However, these defenses are often quickly broken by new and revised attacks. Given the lack of success at generating robust defenses, we are led to ask a fundamental question: Are adversarial attacks inevitable?In this paper, we identify a broad class of problems for which adversarial examples cannot be avoided. We also derive fundamental limits on the susceptibility of a classifier to adversarial attacks that depend on properties of the data distribution as well as the dimensionality of the dataset. Adversarial examples occur when a small perturbation to an image changes its class label. There are different ways of measuring what it means for a perturbation to be "small"; as such, our analysis considers a range of different norms. While the ∞ -norm is commonly used, adversarial examples can be crafted in any p -norm (see FIG0). We will see that the choice of norm can have a dramatic effect on the strength of theoretical guarantees for the existence of adversarial examples. Our analysis also extends to the 0 -norm, which yields "sparse" adversarial examples that only perturb a small subset of image pixels FIG2 ). BID19 on Resnet50, along with the distance between the base image and the adversarial example, and the top class label. 2As a simple example , consider a classification problem with n-dimensional images with pixels scaled between 0 and 1 (in this case images live inside the unit hypercube). If the image classes each occupy a fraction of the cube greater than 1 2 exp(−π 2), then images exist that are susceptible to adversarial perturbations of 2 -norm at most. Note that = 10 was used in FIG0, and larger values are typical for larger images. Finally, in Section 8, we explore the causes of adversarial susceptibility in real datasets, and the effect of dimensionality. We present an example image class for which there is no fundamental link between dimensionality and robustness, and argue that the data distribution, and not dimensionality, is the primary cause of adversarial susceptibility. Adversarial examples, first demonstrated in BID36 and BID3, change the label of an image using small and often imperceptible perturbations to its pixels. A number of defenses have been proposed to harden networks against attacks, but historically, these defenses have been quickly broken. Adversarial training, one of the earliest defenses, successfully thwarted the fast gradient sign method (FGSM) BID13, one of the earliest and simplest attacks. However, adversarial training with FGSM examples was quickly shown to be vulnerable to more sophisticated multi-stage attacks BID16 BID39. More sophisticated defenses that rely on network distillation BID26 and specialized activation functions BID44 were also toppled by strong attacks BID25 BID40 BID6.The ongoing vulnerability of classifiers was highlighted in recent work by BID2 and BID1 that broke an entire suite of defenses presented in ICLR 2018 including thermometer encoding BID5, detection using local intrinsic dimensionality BID18, input transformations such as compression and image quilting BID14, stochastic activation pruning BID10, adding randomization at inference time BID42, enhancing the confidence of image labels BID33, and using a generative model as a defense BID29.Rather than hardening classifiers to attacks, some authors have proposed sanitizing datasets to remove adversarial perturbations before classification. Approaches based on auto-encoders BID22 and GANs BID30 were broken using optimization-based attacks BID8 a).A number of "certifiable" defense mechanisms have been developed for certain classifiers. BID27 harden a two-layer classifier using semidefinite programming, and BID32 propose a convex duality-based approach to adversarial training that works on sufficiently small adversarial perturbations with a quadratic adversarial loss. BID15 consider training a robust classifier using the convex outer adversarial polytope. All of these methods only consider robustness of the classifier on the training set, and robustness properties often fail to generalize reliably to test examples. One place where researchers have enjoyed success is at training classifiers on low-dimensional datasets like MNIST BID19 BID32. The robustness achieved on more complicated datasets such DISPLAYFORM0 Figure 2: Sparse adversarial examples perturb a small subset of pixels and can hide adversarial "fuzz" inside highfrequency image regions. The original image (left) is classified as an "ox." Under ∞-norm perturbations, it is classified as "traffic light", but the perturbations visibly distort smooth regions of the image (the sky). These effects are hidden in the grass using 0-norm (sparse) perturbations limited to a small subset of pixels.as CIFAR-10 and ImageNet are nowhere near that of MNIST, which leads some researchers to speculate that adversarial defense is fundamentally harder in higher dimensions -an issue we address in Section 8.This paper uses well-known from high-dimensional geometry, specifically isoperimetric inequalities, to provide bounds on the robustness of classifiers. Several other authors have investigated adversarial susceptibility through the lens of geometry. BID11 study adversarial susceptibility of datasets under the assumption that they are produced by a generative model that maps random Gaussian vectors onto images. BID12 do a detailed case study, including empirical and theoretical , of classifiers for a synthetic dataset that lies on two concentric spheres. BID31 show that the Lipschitz constant of untrained networks with random weights gets large in high dimensions. Shortly after the original appearance of our work, BID20 presented a study of adversarial susceptibility that included both evasion and poisoning attacks. Our work is distinct in that it studies adversarial robustness for arbitrary data distributions, and also that it rigorously looks at the effect of dimensionality on robustness limits. We use n to denote the unit hypercube in n dimensions, and vol(A) to denote the volume (i.e., ndimensional Lebesgue measure) of a subset A ⊂ n. We use S n−1 = {x ∈ R n | x 2 = 1} to denote the unit sphere embedded in R n, and s n−1 to denote its surface area. The size of a subset A ∈ S n−1 can be quantified by its (n − 1 dimensional) measure µ[A], which is simply the surface area the set covers. Because the surface area of the unit sphere varies greatly with n, it is much easier in practice to work with the normalized measure, which we denote µ 1 [A] = µ[A]/s n−1. This normalized measure has the property that µ 1 [S n−1] = 1, and so we can interpret µ 1 [A] as the probability of a uniform random point from the sphere lying in A. When working with points on a sphere, we often use geodesic distance, which is always somewhat larger than (but comparable to) the Euclidean distance. In the cube, we measure distance between points using p -norms, which are denoted DISPLAYFORM0 Note that · p is not truly a norm for p < 1, but rather a semi-norm. Such metrics are still commonly used, particularly the " 0 -norm" which counts the number of non-zero entries in a vector. We consider the problem of classifying data points that lie in a space Ω (either a sphere or a hypercube) into m different object classes. The m object classes are defined by probability density functions {ρ c} m c=1, where ρ c: Ω → R. A "random" point from class c is a random variable with density ρ c. We assume ρ c to be bounded (i.e., we don't allow delta functions or other generalized functions), and denote its upper bound by U c = sup x ρ c (x).We also consider a "classifier" function C: Ω → {1, 2, . . ., m} that partitions Ω into disjoint measurable subsets, one for each class label. The classifier we consider is discrete valued -it provides a label for each data point but not a confidence level. With this setup, we can give a formal definition of an adversarial example. Definition 1. Consider a point x ∈ Ω drawn from class c, a scalar > 0, and a metric d. We say that x admits an -adversarial example in the metric d if there exists a pointx ∈ Ω with C(x) = c, and d(x,x) ≤.In plain words, a point has an -adversarial example if we can sneak it into a different class by moving it at most units in the distance d. We consider adversarial examples with respect to different p -norm metrics. These metrics are written d p (x,x) = x −x p. A common choice is p = ∞, which limits the absolute change that can be made to any one pixel. However, 2 -norm and 1 -norm adversarial examples are also used, as it is frequently easier to create adversarial examples in these less restrictive metrics. We also consider sparse adversarial examples in which only a small subset of pixels are manipulated. This corresponds to the metric d 0, in which case the constraint x −x 0 ≤ means that an adversarial example was crafted by changing at most pixels, and leaving the others alone. We begin by looking at the case of classifiers for data on the sphere. While this data model may be less relevant than the other models studied below, it provides a straightforward case where can be proven using simple, geometric lemmas. The more realistic case of images with pixels in will be studied in Section 4.The idea is to show that, provided a class of data points takes up enough space, nearly every point in the class lies close to the class boundary. To show this, we begin with a simple definition. Definition 2. The -expansion of a subset A ⊂ Ω with respect to distance metric d, denoted A(, d), contains all points that are at most units away from A. To be precise DISPLAYFORM0 We sometimes simply write A when the distance metric is clear from context. Our provides bounds on the probability of adversarial examples that are independent of the shape of the class boundary. This independence is a simple consequence of an isoperimetric inequality. The classical isoperimetric inequality states that, of all closed surfaces that enclose a unit volume, the sphere has the smallest surface area. This simple fact is intuitive but famously difficult to prove. For a historical review of the isoperimetric inequality and its variants, see BID24. We will use a special variant of the isoperimetric inequality first proved by BID18 and simplified by BID37. Lemma 1 (Isoperimetric inequality). Consider a subset of the sphere A ⊂ S n−1 ⊂ R n with normalized measure µ 1 (A) ≥ 1/2. When using the geodesic metric, the -expansion A is at least as large as the -expansion of a half sphere. The classical isoperimetric inequality is a simple geometric statement, and frequently appears without absolute bounds on the size of the -expansion of a half-sphere, or with bounds that involve unspecified constants BID41. A tight bound derived by BID23 is given below. The asymptotic blow-up of the -expansion of a half sphere predicted by this bound is shown in FIG1. Lemma 2 (-expansion of half sphere). The geodesic -expansion of a half sphere has normalized measure at least DISPLAYFORM1 Lemmas 1 and 2 together can be taken to mean that, if a set is not too small, then in high dimensions almost all points on the sphere are reachable within a short jump from that set. These lemmas have immediate implications for adversarial examples, which are formed by mapping one class into another using small perturbations. Despite its complex appearance, the below is a consequence of the (relatively simple) isoperimetric inequality. Theorem 1 (Existence of Adversarial Examples). Consider a classification problem with m object classes, each distributed over the unit sphere S n−1 ⊂ R n with density functions {ρ c} m c=1. Choose a classifier function C: S n−1 → {1, 2, . . ., m} that partitions the sphere into disjoint measurable subsets. Define the following scalar constants:• Let V c denote the magnitude of the supremum of ρ c relative to the uniform density. This can be written V c:= s n−1 · sup x ρ c (x).• Let f c = µ 1 {x|C(x) = c} be the fraction of the sphere labeled as c by classifier C.Choose some class c with f c ≤ 1 2. Sample a random data point x from ρ c. Then with probability at least DISPLAYFORM2 one of the following conditions holds: 1. x is misclassified by C, or 2. x admits an -adversarial example in the geodesic distance. Proof. Choose a class c with f c ≤ 1 2. Let R = {x|C(x) = c} denote the region of the sphere labeled as class c by C, and let R be its complement. R is the -expansion of R in the geodesic metric. Because R covers at least half the sphere, the isoperimetric inequality (Lemma 1) tells us that the epsilon expansion is at least as great as the epsilon expansion of a half sphere. We thus have DISPLAYFORM3 Now, consider the set S c of "safe" points from class c that are correctly classified and do not admit adversarial perturbations. A point is correctly classified only if it lies inside R, and therefore outside of R. To be safe from adversarial perturbations, a point cannot lie within distance from the class boundary, and so it cannot lie within R. It is clear that the set S c of safe points is exactly the complement of R. This set has normalized measure DISPLAYFORM4 The probability of a random point lying in S c is bounded above by the normalized supremum of ρ c times the normalized measure µ 1 [S c]. This product is given by DISPLAYFORM5 We then subtract this probability from 1 to obtain the probability of a point lying outside the safe region, and arrive at equation 1.In the above , we measure the size of adversarial perturbations using the geodesic distance. Most studies of adversarial examples measure the size of perturbation in either the 2 (Euclidean) norm or the ∞ (max) norm, and so it is natural to wonder whether Theorem 1 depends strongly on the distance metric. Fortunately (or, rather unfortunately) it does not. It is easily observed that, for any two points x and y on a sphere, DISPLAYFORM6 where d ∞ (x, y), d 2 (x, y), and d g (x, y) denote the l ∞, Euclidean, and geodesic distance, respectively. From this, we see that Theorem 1 is actually fairly conservative; any -adversarial example in the geodesic metric would also be adversarial in the other two metrics, and the bound in Theorem 1 holds regardless of which of the three metrics we choose (although different values of will be appropriate depending on the norm). The above about the sphere is simple and easy to prove using classical . However, real world images do not lie on the sphere. In a more typical situation, images will be scaled so that their pixels lie in, and data lies inside a high-dimensional hypercube (but, unlike the sphere, data is not confined to its surface). The proof of Theorem 1 makes extensive use of properties that are exclusive to the sphere, and is not applicable to this more realistic setting. Are there still problem classes on the cube where adversarial examples are inevitable?This question is complicated by the fact that geometric isoperimetric inequalities do not exist for the cube, as the shapes that achieve minimal -expansion (if they exist) depend on the volume they enclose and the choice of BID28. Fortunately, researchers have been able to derive "algebraic" isoperimetric inequalities that provide lower bounds on the size of the -expansion of sets without identifying the shape that achieves this minimum BID38 BID23. The below about the unit cube is analogous to Proposition 2.8 in BID17, except with tighter constants. For completeness, a proof (which utilizes methods from Ledoux) is provided in Appendix A. Lemma 3 (Isoperimetric inequality on a cube). Consider a measurable subset of the cube A ⊂ n, and DISPLAYFORM0 2 /2 dt, and let α be the scalar that satisfies DISPLAYFORM1 where p * = min(p, 2). In particular, if vol(A) ≥ 1/2, then we simply have DISPLAYFORM2 Using this , we can show that most data samples in a cube admit adversarial examples, provided the data distribution is not excessively concentrated. n → {1, 2, . . ., m} that partitions the hypercube into disjoint measurable subsets. Define the following scalar constants:• Let U c denote the supremum of ρ c.• Let f c be the fraction of hypercube partitioned into class c by C.Choose some class c with f c ≤ 1 2, and select an p -norm with p > 0. Define p * = min(p, 2). Sample a random data point x from the class distribution ρ c. Then with probability at least DISPLAYFORM3 one of the following conditions holds:1. x is misclassified by C, or 2. x has an adversarial examplex, with x −x p ≤.When adversarial examples are defined in the 2 -norm (or for any p ≥ 2), the bound in equation 4 becomes DISPLAYFORM4 Provided the class distribution is not overly concentrated, equation 5 guarantees adversarial examples with relatively "small" relative to a typical vector. In n dimensions, the 2 diameter of the cube is √ n, and so it is reasonable to choose = O(√ n) in equation 5. In FIG0, we chose = 10. A similarly strong bound of DISPLAYFORM5 DISPLAYFORM6 2 /2 (for z > 0), and α = Φ −1 (1 − f c). For this bound to be meaningful with < 1, we need f c to be relatively small, and to be roughly f c or smaller. This is realistic for some problems; ImageNet has 1000 classes, and so f c < 10 −3 for at least one class. Interestingly, under ∞ -norm attacks, guarantees of adversarial examples are much stronger on the sphere (Section 3) than on the cube. One might wonder whether the weakness of Theorem 4 in the ∞ case is fundamental, or if this is a failure of our approach. One can construct examples of sets with ∞ expansions that nearly match the behavior of equation 5, and so our theorems in this case are actually quite tight. It seems to be inherently more difficult to prove the existence of adversarial examples in the cube using the ∞ -norm. A number of papers have looked at sparse adversarial examples, in which a small number of image pixels, in some cases only one BID34, are changed to manipulate the class label. To study this case, we would like to investigate adversarial examples under the 0 metric. The 0 distance is defined as DISPLAYFORM0 If a point x has an -adversarial example in this norm, then it can be perturbed into a different class by modifying at most pixels (in this case is taken to be a positive integer).Theorem 2 is fairly tight for p = 1 or 2. However, the bound becomes quite loose for small p, and in particular it fails completely for the important case of p = 0. For this reason, we present a different bound that is considerably tighter for small p (although slightly looser for large p).The case p = 0 was studied by (Section 6.2) and BID21, and later by BID37 BID38. The proof of the following theorem (appendix B) follows the method used in Section 5 of BID38, with modifications made to extend the proof to arbitrary p. Lemma 4 (Isoperimetric inequality on the cube: small p). Consider a measurable subset of the cube A ⊂ n, and a p-norm distance metric d(x, y) = x − y p for any p ≥ 0. We have DISPLAYFORM1 Using this , we can prove a statement analogous to Theorem 2, but for sparse adversarial examples. We present only the case of p = 0, but the generalization to the case of other small p using Lemma 4 is straightforward. Theorem 3 (Sparse adversarial examples). Consider the problem setup of Theorem 2. Choose some class c with f c ≤ 1 2, and sample a random data point x from the class distribution ρ c. Then with probability at least DISPLAYFORM2 one of the following conditions holds: 1. x is misclassified by C, or 2. x can be adversarially perturbed by modifying at most pixels, while still remaining in the unit hypercube. Tighter bounds can be obtained if we only guarantee that adversarial examples exist for some data points in a class, without bounding the probability of this event. Theorem 4 (Condition for existence of adversarial examples). Consider the setup of Theorem 2. Choose a class c that occupies a fraction of the cube f c < 1 2. Pick an p norm and set p * = min(p, 2).Let supp(ρ c) denote the support of ρ c. Then there is a point x with ρ c (x) > 0 that admits an -adversarial example if DISPLAYFORM0 The bound for the case p = 0 is valid only if ≥ n log 2/2.It is interesting to consider when Theorem 4 produces non-vacuous bounds. When the 2 -norm is used, the bound becomes vol[supp(ρ c)] ≥ exp(−π 2)/2. The diameter of the cube is √ n, and so the bound becomes active for = √ n. Plugging this in, we see that the bound is active whenever the size of the support satisfies DISPLAYFORM1 2e πn. Remarkably, this holds for large n whenever the support of class c is larger than (or contains) a hypercube of side length at least e −π ≈ 0.043. Note, however, that the bound being "active" does not guarantee adversarial examples with a "small". There are a number of ways to escape the guarantees of adversarial examples made by Theorems 1-4. One potential escape is for the class density functions to take on extremely large values (i.e., exponentially large U c); the dependence of U c on n is addressed separately in Section 8.Unbounded density functions and low-dimensional data manifolds In practice, image datasets might lie on low-dimensional manifolds within the cube, and the support of these distributions could have measure zero, making the density function infinite (i.e., U c = ∞). The arguments above are still relevant (at least in theory) in this case; we can expand the data manifold by adding a uniform random noise to each image pixel of magnitude at most 1. The expanded dataset has positive volume. Then, adversarial examples of this expanded dataset can be crafted with perturbations of size 2. This method of expanding the manifold before crafting adversarial examples is often used in practice. BID39 proposed adding a small perturbation to step off the image manifold before crafting adversarial examples. This strategy is also used during adversarial training BID19.Adding a "don't know" class The analysis above assumes the classifier assigns a label to every point in the cube. If a classifier has the ability to say "I don't know," rather than assign a label to every input, then the region of the cube that is assigned class labels might be very small, and adversarial examples could be escaped even if the other assumptions of Theorem 4 are satisfied. In this case, it would still be easy for the adversary to degrade classifier performance by perturbing images into the "don't know" class. Feature squeezing If decreasing the dimensionality of data does not lead to substantially increased values for U c (we see in Section 8 that this is a reasonable assumption) or loss in accuracy (a stronger assumption), measuring data in lower dimensions could increase robustness. This can be done via an auto-encoder BID22 BID30, JPEG encoding BID9, or quantization BID43.Computational hardness It may be computationally hard to craft adversarial examples because of local flatness of the classification function, obscurity of the classifier function, or other computational difficulties. Computational hardness could prevent adversarial attacks in practice, even if adversarial examples still exist. In this section, we discuss the relationship between dimensionality and adversarial robustness, and explore how the predictions made by the theorems above are reflected in experiments. It is commonly thought that high-dimensional classifiers are more susceptible to adversarial examples than low-dimensional classifiers. This perception is partially motivated by the observation that classifiers on highresolution image distributions like ImageNet are more easily fooled than low resolution classifiers on MNIST BID39. Indeed, Theorem 2 predicts that high-dimensional classifiers should be much easier to fool than low-dimensional classifiers, assuming the datasets they classify have comparable probability density limits U c. However, this is not a reasonable assumption; we will see below that high dimensional distributions may be more concentrated than their low-dimensional counterparts. We study the effects of dimensionality with a thought experiment involving a "big MNIST" image distribution. Given an integer expansion factor b, we can make a big MNIST distribution, denoted b-MNIST, by replacing each pixel in an MNIST image with a b × b array of identical pixels. This expands an original 28 × 28 image into a 28b × 28b image. FIG3 shows that, without adversarial training, a classifier on big MNIST is far more susceptible to attacks than a classifier trained on the original MNIST 1.However, each curve in FIG3 only shows the attack susceptibility of one particular classifier. In contrast, Theorems 1-4 describe the fundamental limits of susceptibility for all classifiers. These limits are an inherent property of the data distribution. The theorem below shows that these fundamental limits do not depend in a non-trivial way on the dimensionality of the images in big MNIST, and so the relationship between dimensionality and susceptibility in FIG3 from the weakness of the training process. Theorem 5. Suppose and p are such that, for all MNIST classifiers, a random image from class c has an -adversarial example (in the 2 -norm) with probability at least p. Then for all classifiers on b-MNIST, with integer b ≥ 1, a random image from c has a b -adversarial example with probability at least p. Likewise, if all b-MNIST classifiers have b -adversarial examples with probability p for some b ≥ 1, then all classifiers on the original MNIST distribution have -adversarial examples with probability p. Theorem 5 predicts that the perturbation needed to fool all 56 × 56 classifiers is twice that needed to fool all 28 × 28 classifiers. This is reasonable since the 2 -norm of a 56 × 56 image is twice that of its 28 × 28 counterpart. Put simply, fooling big MNIST is just as hard/easy as fooling the original MNIST regardless of resolution. This also shows that for big MNIST, as the expansion factor b gets larger and is expanded to match, the concentration bound U c grows at exactly the same rate as the exponential term in equation 2 shrinks, and there is no net effect on fundamental susceptibility. Also note that an analogous could be based on any image classification problem (we chose MNIST only for illustration), and any p ≥ 0.We get a better picture of the fundamental limits of MNIST by considering classifiers that are hardened by adversarial training 2 FIG3 ). These curves display several properties of fundamental limits predicted by our theorems. As predicted by Theorem 5, the 112 × 112 classifer curve is twice as wide as the 56 × 56 curve, which in turn is twice as wide as the 28×28 curve. In addition, we see the kind of "phase transition" behavior predicted by Theorem 2, in which the classifier suddenly changes from being highly robust to being highly susceptible as passes a critical threshold. For these reasons, it is reasonable to suspect that the adversarially trained classifiers in FIG3 are operating near the fundamental limit predicted by Theorem 2.Theorem 5 shows that increased dimensionality does not increase adversarial susceptibility in a fundamental way. But then why are high-dimensional classifiers so easy to fool? To answer this question, we look at the concentration bound U c for object classes. The smallest possible value of U c is 1, which only occurs when images are "spread out" with uniform, uncorrelated pixels. In contrast, adjacent pixels in MNIST (and especially big MNIST) are very highly correlated, and images are concentrated near simple, low-dimensional manifolds, ing in highly concentrated image classes with large U c. Theory predicts that such highly concentrated datasets can be relatively safe from adversarial examples. Under review as a conference paper at ICLR 2019 We can reduce U c and dramatically increase susceptibility by choosing a more "spread out" dataset, like CIFAR-10, in which adjacent pixels are less strongly correlated and images appear to concentrate near complex, higher-dimensional manifolds. We observe the effect of decreasing U c by plotting the susceptibility of a 56 × 56 MNIST classifier against a classifier for CIFAR-10 FIG3, right). The former problem lives in 3136 dimensions, while the latter lives in 3072, and both have 10 classes. Despite the structural similarities between these problems, the decreased concentration of CIFAR-10 in vastly more susceptibility to attacks, regardless of whether adversarial training is used. The theory above suggests that this increased susceptibility is caused at least in part by a shift in the fundamental limits for CIFAR-10, rather than the weakness of the particular classifiers we chose. Informally, the concentration limit U c can be interpreted as a measure of image complexity. Image classes with smaller U c are likely concentrated near high-dimensional complex manifolds, have more intra-class variation, and thus more apparent complexity. An informal interpretation of Theorem 2 is that "high complexity" image classes are fundamentally more susceptible to adversarial examples, and FIG3 suggests that complexity (rather than dimensionality) is largely responsible for differences we observe in the effectiveness of adversarial training for different datasets. The question of whether adversarial examples are inevitable is an ill-posed one. Clearly, any classification problem has a fundamental limit on robustness to adversarial attacks that cannot be escaped by any classifier. However, we have seen that these limits depend not only on fundamental properties of the dataset, but also on the strength of the adversary and the metric used to measure perturbations. This paper provides a characterization of these limits and how they depend on properties of the data distribution. Unfortunately, it is impossible to know the exact properties of real-world image distributions or the ing fundamental limits of adversarial training for specific datasets. However, the analysis and experiments in this paper suggest that, especially for complex image classes in high-dimensional spaces, these limits may be far worse than our intuition tells us. A PROOF OF LEMMA 3We now prove Lemma 3. To do this, we begin with a classical isoperimetric inequality for random Gaussian variables. Unlike the case of a cube, tight geometric isoperimetric inequalities exist in this case. We then prove about the cube by creating a mapping between uniform random variables on the cube and random Gaussian vectors. In the lemma below, we consider the standard Gaussian density in R n given by p(x) = 1 (2π) n/2 e −nx 2 /2 and corresponding Gaussian measure µ. We also define DISPLAYFORM0 which is the cumulative density of a Gaussian curve. The following Lemma was first proved in BID35, and an elementary proof was given in BID4. Lemma 5 (Gaussian Isoperimetric Inequality). Of all sets with the same Gaussian measure, the set with 2 -expansion of smallest measure is a half space. Furthermore, for any measurable set A ⊂ R n, and scalar constant a such that DISPLAYFORM1 Using this we can now give a proof of Lemma 3.This function Φ maps a random Guassian vector z ∈ N (0, I) onto a random uniform vector in the unit cube. To see why, consider a measurable subset B ⊂ R n. If µ is the Gaussian measure on R n and σ is the uniform measure on the cube, then DISPLAYFORM2. DISPLAYFORM3, we also have DISPLAYFORM4 for any z, w ∈ R n. From this, we see that for p * = min(p, 2) DISPLAYFORM5 where we have used the identity u p ≤ n 1/ min(p,2)−1/2 u 2. Now, consider any set A in the cube, and let B = Φ −1 (A). From equation 10, we see that DISPLAYFORM6 It follows from equation 10 that DISPLAYFORM7 Applying Lemma 5, we see that DISPLAYFORM8 where DISPLAYFORM9 To obtain the simplified formula in the theorem, we use the identity DISPLAYFORM10 which is valid for x > 0, and can be found in BID0.B PROOF OF LEMMA 4Our proof emulates the method of Talagrand, with minor modifications that extend the to other p norms. We need the following standard inequality. Proof can be found in BID37 BID38. Lemma 6 (Talagrand). Consider a probability space Ω with measure µ. For g: Ω →, we have DISPLAYFORM11 Our proof of Lemma 3 follows the three-step process of Talagrand illustrated in BID37. We begin by proving the bound DISPLAYFORM12 where DISPLAYFORM13 is a measure of distance from A to x, and α, t are arbitrary positive constants. Once this bound is established, a Markov bound can be used to obtain the final . Finally, constants are tuned in order to optimize the tightness of the bound. We start by proving the bound in equation 12 using induction on the dimension. The base case for the induction is n = 1, and we have DISPLAYFORM14 We now prove the for n dimensions using the inductive hypothesis. We can upper bound the integral by integrating over "slices" along one dimension. Let A ⊂ n. Define DISPLAYFORM15 Clearly, the distance from (ω, z) to A is at most the distance from z to A ω, and so DISPLAYFORM16 We also have that the distance from x to A is at most one unit greater than the distance from x to B. This gives us. DISPLAYFORM17 Now, we apply lemma 6 to equation 13 with g(ω) = α[A ω]/α[B] to arrive at equation 12.The second step of the proof is to produce a Markov inequality from equation 12. For the bound in equation 12 to hold, we need DISPLAYFORM18 The third step is to optimize the bound by choosing constants. We minimize the right hand side by choosing t = Now, we can simply choose α = 1 to get the simple bound DISPLAYFORM19 or we can choose the optimal value of α = 2 2p n log(1/σ) − 1, which optimizes the bound in the case 2p ≥ n 2 log(1/σ(A)). We arrive at DISPLAYFORM20 This latter bound is stronger than we need to prove Lemma 3, but it will come in handy later to prove Theorem 4.C PROOF OF THEOREMS 2 AND 3We combine the proofs of these since their proofs are nearly identical. The proofs closely follow the argument of Theorem 1.Choose a class c with f c ≤ 1 2 and let R = {x|C(x) = c} denote the subset of the cube lying in class c according to the classifier C. Let R be the complement, who's p expansion is denoted R(; d p). Because R covers at least half the cube, we can invoke Lemma 3. We have that vol[R( ; h)] ≥ 1 − δ, where δ = exp(−πn 1−2/p * 2) 2πn 1/2−1/p *, for Theorem 2 and 2U c exp(− 2 /n), for Theorem 3.The set R(; h) contains all points that are correctly classified and safe from adversarial perturbations. This region has volume at most δ, and the probability of a sample from the class distribution ρ c lying in this region is at most U c δ. We then subtract this from 1 to obtain the mass of the class distribution lying in the "unsafe" region R c. Let A denote the support of p c, and suppose that this support has measure vol[A] = η. We want to show that, for large enough, the expansion A(, d p) is larger than half the cube. Since class c occupies less than half the cube, this would imply that A(, d p) overlaps with other classes, and so there must be data points in A with -adversarial examples. We start with the case p > 0, where we bound A(, d p) using equation 2 of Lemma 3. To do this, we need to approximate Φ −1 (η). This can be done using the inequality Φ(α) = 1 2π The quantity on the left will be greater than This can be re-arranged to obtain the desired . In the case p = 0, we need to use equation 17 from the proof of Lemma 3 in Appendix B, which we restate here

Represent this text to find a much much longer article about it with say hundreds-thousands of words

This paper identifies classes of problems for which adversarial examples are inescapable, and derives fundamental bounds on the susceptibility of any classifier to adversarial examples.

Represent the TLDR for finding the corresponding scientific article

This paper 1) characterizes functions representable by ReLU DNNs, 2) formally studies the benefit of depth in such architectures,  3) gives an algorithm to implement empirical risk minimization to global optimality for two layer ReLU nets.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: For computer vision applications, prior works have shown the efficacy of reducing numeric precision of model parameters (network weights) in deep neural networks. Activation maps, however, occupy a large memory footprint during both the training and inference step when using mini-batches of inputs. One way to reduce this large memory footprint is to reduce the precision of activations. However, past works have shown that reducing the precision of activations hurts model accuracy. We study schemes to train networks from scratch using reduced-precision activations without hurting accuracy. We reduce the precision of activation maps (along with model parameters) and increase the number of filter maps in a layer, and find that this scheme matches or surpasses the accuracy of the baseline full-precision network. As a , one can significantly improve the execution efficiency (e.g. reduce dynamic memory footprint, memory band- width and computational energy) and speed up the training and inference process with appropriate hardware support. We call our scheme WRPN -- wide reduced-precision networks. We report and show that WRPN scheme is better than previously reported accuracies on ILSVRC-12 dataset while being computationally less expensive compared to previously reported reduced-precision networks. A promising approach to lower the compute and memory requirements of convolutional deeplearning workloads is through the use of low numeric precision algorithms. Operating in lower precision mode reduces computation as well as data movement and storage requirements. Due to such efficiency benefits, there are many existing works which propose low-precision deep neural networks (DNNs) BID27 BID12 BID14 BID6; BID24, even down to 2-bit ternary mode BID29 BID11 BID25 and 1-bit binary mode BID28 BID2 BID16 BID23. However, majority of existing works in low-precision DNNs sacrifice accuracy over the baseline full-precision networks. Further, most prior works target reducing the precision of the model parameters (network weights). This primarily benefits the inference step only when batch sizes are small. We observe that activation maps (neuron outputs) occupy more memory compared to the model parameters for batch sizes typical during training. This observation holds even during inference when batch size is around eight or more. Based on this observation, we study schemes for training and inference using low-precision DNNs where we reduce the precision of activation maps as well as the model parameters without sacrificing network accuracy. To improve both execution efficiency and accuracy of low-precision networks, we reduce both the precision of activation maps and model parameters and increase the number of filter maps in a layer. We call networks using this scheme wide reduced-precision networks (WRPN) and find that this scheme compensates or surpasses the accuracy of the baseline full-precision network. Although the number of raw compute operations increases as we increase the number of filter maps in a layer, the compute bits required per operation is now a fraction of what is required when using full-precision operations (e.g. going from FP32 AlexNet to 4-bits precision and doubling the number of filters increases the number of compute operations by 4x, but each operation is 8x more efficient than FP32). WRPN offers better accuracies, while being computationally less expensive compared to previously reported reduced-precision networks. We report on AlexNet BID10, batch-normalized Inception BID8, and ResNet-34 BID7 on ILSVRC-12 dataset. We find 4-bits to be sufficient for training deep and wide models while achieving similar or better accuracy than baseline network. With 4-bit activation and 2-bit weights, we find the accuracy to be at-par with baseline full-precision. Making the networks wider and operating with 1-bit precision, we close the accuracy gap between previously reported binary networks and show state-of-the art for ResNet-34 (69.85% top-1 with 2x wide) and AlexNet (48.04% top-1 with 1.3x wide). To the best of our knowledge, our reported accuracies with binary networks and 4-bit precision are highest to date. Our reduced-precision quantization scheme is hardware friendly allowing for efficient hardware implementations. To this end, we evaluate efficiency benefits of low-precision operations (4-bits to 1-bits) on Titan X GPU, Arria-10 FPGA and ASIC. We see that FPGA and ASIC can deliver significant efficiency gain over FP32 operations (6.5x to 100x), while GPU cannot take advantage of very low-precision operations. While most prior works proposing reduced-precision networks work with low precision weights (e.g. work in BID2 BID29 BID28 BID25 ; BID11 ; BID23), we find that activation maps occupy a larger memory footprint when using mini-batches of inputs. Using mini-batches of inputs is typical in training of DNNs and cloud-based batched inference BID9. FIG0 shows memory footprint of activation maps and filter maps as batch size changes for 4 different networks (AlexNet, Inception-Resnet-v2 BID22, during the training and inference steps. Figure 2: Memory requirements of a feed forward convolutional deep neural network. Orange boxes denote weights (W), blue boxes are activations (ACT) and green boxes are gradient-maps (Grad).As batch-size increases, because of filter reuse across batches of inputs, activation maps occupy significantly larger fraction of memory compared to the filter weights. This aspect is illustrated in Figure 2 which shows the memory requirements of a canonical feed-forward DNN for a hardware accelerator based system (e.g. GPU, FPGA, PCIe connected ASIC device, etc.). During training, the sum of all the activation maps (ACT) and weight tensors (W) are allocated in device memory for forward pass along with memory for gradient maps during backward propagation. The total memory requirements for training phase is the sum of memory required for the activation maps, weights and the maximum of input gradient maps (δZ) and maximum of back-propagated gradients (δX). During inference, memory is allocated for input (IFM) and output feature maps (OFM) required by a single layer, and these memory allocations are reused for other layers. The total memory allocation during inference is then the maximum of IFM and maximum of OFM required across all the layers plus the sum of all W-tensors. At batch sizes 128 and more, activations start to occupy more than 98% of total memory footprint during training. Overall, reducing precision of activations and weights reduces memory footprint, bandwidth and storage while also simplifying the requirements for hardware to efficiently support these operations.3 WRPN SCHEME AND STUDIES ON ALEXNET Based on the observation that activations occupy more memory footprint compared to weights, we reduce the precision of activations to speed up training and inference steps as well as cut down on memory requirements. However, a straightforward reduction in precision of activation maps leads to significant reduction in model accuracy BID28 BID16.We conduct a sensitivity study where we reduce precision of activation maps and model weights for AlexNet running ILSVRC-12 dataset and train the network from scratch. BID16. 32bA and 2bW data-point in this table is using Trained Ternary Quantization (TTQ) technique BID29. All other data points are collected using our quantization scheme (described later in Section 5), all the runs have same hyper-parameters and training is carried out for the same number of epochs as baseline network. To be consistent with reported in prior works, we do not quantize weights and activations of the first and last layer. We find that, in general, reducing the precision of activation maps and weights hurts model accuracy. Further, reducing precision of activations hurts model accuracy much more than reducing precision of the filter parameters. We find TTQ to be quite effective on AlexNet in that one can lower the precision of weights to 2b (while activations are still FP32) and not lose accuracy. However, we did not find this scheme to be effective for other networks like ResNet or Inception. To re-gain the model accuracy while working with reduced-precision operands, we increase the number of filter maps in a layer. Although the number of raw compute operations increase with widening the filter maps in a layer, the bits required per compute operation is now a fraction of what is required when using full-precision operations. As a , with appropriate hardware support, one can significantly reduce the dynamic memory requirements, memory bandwidth, computational energy and speed up the training and inference process. Our widening of filter maps is inspired from Wide ResNet BID26 work where the depth of the network is reduced and width of each layer is increased (the operand precision is still FP32). Wide ResNet requires a re-design of the network architecture. In our work, we maintain the depth parameter same as baseline network but widen the filter maps. We call our approach WRPN -wide reduced-precision networks. In practice, we find this scheme to be very simple and effective -starting with a baseline network architecture, one can change the width of each filter map without changing any other network design parameter or hyper-parameters. Carefully reducing precision and simultaneously widening filters keeps the total compute cost of the network under or at-par with baseline cost.1 TAB2 reports the accuracy of AlexNet when we double the number of filter maps in a layer. With doubling of filter maps, AlexNet with 4-bits weights and 2-bits activations exhibits accuracy at-par with full-precision networks. Operating with 4-bits weights and 4-bits activations surpasses the baseline accuracy by 1.44%. With binary weights and activations we better the accuracy of XNOR-NET BID16 by 4%.When doubling the number of filter maps, AlexNet's raw compute operations grow by 3.9x compared to the baseline full-precision network, however by using reduced-precision operands the overall compute complexity is a fraction of the baseline. For example, with 4b operands for weights and activations and 2x the number of filters, reduced-precision AlexNet is just 49% of the total compute cost of the full-precision baseline (compute cost comparison is shown in TAB3). We also experiment with other widening factors. With 1.3x widening of filters and with 4-bits of activation precision one can go as low as 8-bits of weight precision while still being at-par with baseline accuracy. With 1.1x wide filters, at least 8-bits weight and 16-bits activation precision is required for accuracy to match baseline full-precision 1x wide accuracy. Further, as TAB3 shows, when widening filters by 2x, one needs to lower precision to at least 8-bits so that the total compute cost is not more than baseline compute cost. Thus, there is a trade-off between widening and reducing the precision of network parameters. In our work, we trade-off higher number of raw compute operations with aggressively reducing the precision of the operands involved in these operations (activation maps and filter weights) while not sacrificing the model accuracy. Apart from other benefits of reduced precision activations as mentioned earlier, widening filter maps also improves the efficiency of underlying GEMM calls for convolution operations since compute accelerators are typically more efficient on a single kernel consisting of parallel computation on large data-structures as opposed to many small sized kernels BID26. We study how our scheme applies to deeper networks. For this, we study ResNet-34 BID7 and batch-normalized Inception BID8 and find similar trends, particularly that 2-bits weight and 4-bits activations continue to provide at-par accuracy as baseline. We use TensorFlow BID0 and tensorpack for all our evaluations and use ILSVRC-12 train and val dataset for analysis. ResNet-34 has 3x3 filters in each of its modular layers with shortcut connections being 1x1. The filter bank width changes from 64 to 512 as depth increases. We use the pre-activation variant of ResNet and the baseline top-1 accuracy of our ResNet-34 implementation using single-precision 32-bits data format is 73.59%. Binarizing weights and activations for all layers except the first and the last layer in this network gives top-1 accuracy of 60.5%. For binarizing ResNet we did not re-order any layer (as is done in XNOR-NET). We used the same hyper-parameters and learning rate schedule as the baseline network. As a reference, for ResNet-18, the gap between XNOR-NET (1b weights and activations) and full-precision network is 18% BID16. It is also interesting to note that top-1 accuracy of single-precision AlexNet (57.20%) is lower than the top-1 accuracy of binarized ResNet-34 (60.5%). We experimented with doubling number of filters in each layer and reduce the precision of activations and weights. TAB4 shows the of our analysis. Doubling the number of filters and 4-bits precision for both weights and activations beats the baseline accuracy by 0.9%. 4-bits activations and 2-bits (ternary) weights has top-1 accuracy at-par with baseline. Reducing precision to 2-bits for both weights and activations degrades accuracy by only 0.2% compared to baseline. Binarizing the weights and activations with 2x wide filters has a top-1 accuracy of 69.85%. This is just 3.7% worse than baseline full-precision network while being only 15% of the cost of the baseline network. Widening the filters by 3x and binarizing the weights and activations reduces this gap to 1.2% while the 3x wide network is 30% the cost of the full-precision baseline network. Although 4-bits precision seems to be enough for wide networks, we advocate for 4-bits activation precision and 2-bits weight precision. This is because with ternary weights one can get rid of the multipliers and use adders instead. Additionally, with this configuration there is no loss of accuracy. Further, if some accuracy degradation is tolerable, one can even go to binary circuits for efficient hardware implementation while saving 32x in bandwidth for each of weights and activations compared to full-precision networks. All these gains can be realized with simpler hardware implementation and lower compute cost compared to baseline networks. To the best of our knowledge, our ResNet binary and ternary (with 2-bits or 4-bits activation) top-1 accuracies are state-of-the-art in the literature including unpublished technical reports (with similar data augmentation BID13). We applied WRPN scheme to batch-normalized Inception network BID8. This network includes batch normalization of all layers and is a variant of GoogleNet BID21 where the 5x5 convolutional filters are replaced by two 3x3 convolutions with up to 128 wide filters. TAB5 shows the of our analysis. Using 4-bits activations and 2-bits weight and doubling the number of filter banks in the network produces a model that is almost at-par in accuracy with the baseline single-precision network (0.02% loss in accuracy). Wide network with binary weights and activations is within 6.6% of the full-precision baseline network. We adopt the straight-through estimator (STE) approach in our work BID1. When quantizing a real number to k-bits, the ordinality of the set of quantized numbers is 2 k. Mathemat- ically, this small and finite set would have zero gradients with respect to its inputs. STE method circumvents this problem by defining an operator that has arbitrary forward and backward operations. Prior works using the STE approach define operators that quantize the weights based on the expectation of the weight tensors. For instance, Ternary Weight Networks (TWN) BID11 ) uses a threshold and a scaling factor for each layer to quantize weights to ternary domain. In TTQ BID29, the scaling factors are learned parameters. XNOR-NET binarizes the weight tensor by computing the sign of the tensor values and then scaling by the mean of the absolute value of each output channel of weights. DoReFa uses a single scaling factor across the entire layer. For quantizing weights to k-bits, where k > 1, DoReFa uses: DISPLAYFORM0 Here w k is the k-bit quantized version of inputs w i and quantize k is a quantization function that quantizes a floating-point number w i in the range to a k-bit number in the same range. The transcendental tanh operation constrains the weight value to lie in between −1 and +1. The affine transformation post quantization brings the range to [−1, 1].We build on these approaches and propose a much simpler scheme. For quantizing weight tensors we first hard constrain the values to lie within the range [−1, 1] using min-max operation (e.g. tf.clip by val when using Tensorflow BID0). For quantizing activation tensor values, we constrain the values to lie within the range. This step is followed by a quantization step where a real number is quantized into a k-bit number. This is given as, for k > 1: DISPLAYFORM1 Here w i and a i are input real-valued weights and activation tensor and w k and a k are their quantized versions. One bit is reserved for sign-bit in case of weight values, hence the use of 2 k−1 for these quantized values. Thus, weights can be stored and interpreted using signed data-types and activations using un-signed data-types. With appropriate affine transformations, the convolution operations (the bulk of the compute operations in the network during forward pass) can be done using quantized values (integer operations in hardware) followed by scaling with floating-point constants (this scaling operation can be done in parallel with the convolution operation in hardware). When k = 1, for binary weights we use the Binary Weighted Networks (BWN) approach where the binarized weight value is computed based on the sign of input value followed by scaling with the mean of absolute values. For binarized activations we use the formulation in Eq. 2. We do not quantize the gradients and maintain the weights in reduced precision format. For convolution operation when using WRPN, the forward pass during training (and the inference step) involves matrix multiplication of k-bits signed and k-bits unsigned operands. Since gradient values are in 32-bits floating-point format, the backward pass involves a matrix multiplication operation using 32-bits and k-bits operand for gradient and weight update. When k > 1, the hard clipping of tensors to a range maps efficiently to min-max comparator units in hardware as opposed to using transcendental operations which are long latency operations. TTQ and DoRefa BID28 schemes involve division operation and computing a maximum value in the input tensor. Floating-point division operation is expensive in hardware and computing the maximum in a tensor is an O(n) operation. Additionally, our quantization parameters are static and do not require any learning or involve back-propagation like TTQ approach. We avoid each of these costly operations and propose a simpler quantization scheme (clipping followed by rounding). In practice, the effective performance and energy efficiency one could achieve on a low-precision compute operation highly depends on the hardware that runs these operations. We study the efficiency of low-precision operations on various hardware targets GPU, FPGA, and ASIC.For GPU, we evaluate WRPN on Nvidia Titan X Pascal and for FPGA we use Intel Arria-10. We collect performance numbers from both previously reported analysis BID15 as well as our own experiments. For FPGA, we implement a DNN accelerator architecture shown in FIG2 (a). This is a prototypical accelerator design used in various works (e.g., on FPGA and ASIC such as TPU BID9 ). The core of the accelerator consists of a systolic array of processing elements (PEs) to perform matrix and vector operations, along with on-chip buffers, as well as off-chip memory management unit. The PEs can be configured to support different precision -(FP32, FP32), (INT4, INT4), (INT4, TER2), and (BIN1, BIN1). The (INT4, TER2) PE operates on ternary (+1,0,-1) values and is optimized to include only an adder since there is no need for a multiplier in this case. The binary (BIN1, BIN1) PE is implemented using XNOR and bitcount. Our RTL design targets Arria-10 1150 FPGA. For our ASIC study, we synthesize the PE design using Intel 14 nm process technology to obtain area and energy estimates. FIG2 shows the efficiency improvements using first-order estimates where the efficiency is computed based on number of bits used in the operation. DISPLAYFORM0 With this method we would expect (INT4, INT4) and (BIN1, BIN1) to be 8x and 32x more efficient, respectively, than (FP32, FP32). However, in practice the efficiency gains from reducing precision depend on whether the underlying hardware can take advantage of such low-precisions. Figure 3(c) shows performance improvement on Titan X GPU for various low-precision operations relative to FP32. In this case, GPU can only achieve up to ∼4x improvements in performance over FP32 baseline. This is because GPU only provides first-class support for INT8 operations, and is not able to take advantage of the lower INT4, TER2, and BIN1 precisions. On the contrary, FPGA can take advantage of such low precisions, since they are amenable for implementations on the FPGAs reconfigurable fabric.. In fact, for (BIN1, BIN1), FPGA improvements exceed the first-order estimate. Reducing the precision simplifies the design of compute units and lower buffering requirements on FPGA board. Compute-precision reduction leads to significant improvement in throughput due to smaller hardware designs (allowing more parallelism) and shorter circuit delay (allowing higher frequency). FIG2 shows the performance and performance/Watt of the reduced-precision operations on GPU and FPGA. FPGA performs quite well on very low precision operations. In terms of performance/watt, FPGA does better than GPU on (INT4, INT4) and lower precisions. ASIC allows for a truly customized hardware implementation. Our ASIC study provides insights to the upper bound of the efficiency benefits possible from low-precision operations. FIG2 (f) and 3(g) show improvement in performance and energy efficiency of the various low-precision ASIC PEs relative to baseline FP32 PE. As the figures show, going to lower precision offers 2 to 3 orders of magnitude efficiency improvements. In summary, FPGA and ASIC are well suited for our WRPN approach. At 2x wide, our WRPN approach requires 4x more total operations than the original network. However, for INT4 or lower precision, each operation is 6.5x or better in efficiency than FP32 for FPGA and ASIC. Hence, WRPN delivers an overall efficiency win. Reduced-precision DNNs is an active research area. Reducing precision of weights for efficient inference pipeline has been very well studied. Works like Binary connect (BC), Ternary-weight networks (TWN) BID11, fine-grained ternary quantization BID13 and INQ BID27 target precision reduction of network weights while still using full-precision activations. Accuracy is almost always degraded when quantizing the weights. For AlexNet on Imagenet, TWN loses 5% top-1 accuracy. Schemes like INQ, BID20 and BID13 do fine-tuning to quantize the network weights and do not sacrifice accuracy as much but are not applicable for training networks from scratch. INQ shows promising with 5-bits of precision. XNOR-NET BID16, BNN BID2, DoReFa BID28 and TTQ BID29 target training as well. While TTQ targets weight quantization only, most works targeting activation quantization hurt accuracy. XNOR-NET approach reduces top-1 accuracy by 12% and DoReFa by 8% when quantizing both weights and activations to 1-bit (for AlexNet on ImageNet). Further, XNOR-NET requires re-ordering of layers for its scheme to work. Recent work in BID4 targets low-precision activations and reports accuracy within 1% of baseline with 5-bits precision and logarithmic (with base √ 2) quantization. With fine-tuning this gap can be narrowed to be within 0.6% but not all layers are quantized. Non-multiples of two for operand values introduces hardware inefficiency in that memory accesses are no longer DRAM or cache-boundary aligned and end-to-end run-time performance aspect is unclear when using complicated quantization schemes. We target end-to-end training and inference, using very simple quantization method and aim for reducing precision without any loss in accuracy. To the best of our knowledge, our work is the first to study reduced-precision deep and wide networks, and show accuracy at-par with baseline for as low a precision as 4-bits activations and 2-bits weights. We report state of the art accuracy for wide binarized AlexNet and ResNet while still being lower in compute cost. Work by BID5 advocates for low precision fixed-point numbers for training. They show 16-bits to be sufficient for training on CIFAR10 dataset and find stochastic rounding to be necessary for training convergence. In our work here we focus on sub-8b training and like DoReFa scheme do not see stochastic rounding necessary when using full-precision gradients. Work by BID19 quantizes gradients before communication in a distributed computing setting. They use full precision gradients during the backward pass and quantize the gradients before sending them to other computation nodes (decreasing the amount of communication traffic over an interconnection network). For distributed training, we can potentially use this approach for communicating gradients across nodes. We present the Wide Reduced-Precision Networks (WRPN) scheme for DNNs. In this scheme, the numeric precision of both weights and activations are significantly reduced without loss of network accuracy. This is in contrast to many previous works that find reduced-precision activations to detrimentally impact accuracy; specifically, we find that 2-bit weights and 4-bit activations are sufficient to match baseline accuracy across many networks including AlexNet, ResNet-34 and batchnormalized Inception. We achieve this with a new quantization scheme and by increasing the number of filter maps in each reduced-precision layer to compensate for the loss of information capacity induced by reducing the precision. We believe ours to be the first work to study the interplay between layer width and precision -with widening, the number of neurons in a layer increase; yet with reduced precision, we control overfitting and regularization. We motivate this work with our observation that full-precision activations contribute significantly more to the memory footprint than full-precision weight parameters when using mini-batch sizes common during training and cloud-based inference; furthermore, by reducing the precision of both activations and weights the compute complexity is greatly reduced (40% of baseline for 2-bit weights and 4-bit activations).The WRPN quantization scheme and computation on low precision activations and weights is hardware friendly making it viable for deeply-embedded system deployments as well as in cloud-based training and inference servers with compute fabrics for low-precision. We compare Titan X GPU, Arria-10 FPGA and ASIC implementations using WRPN and show our scheme increases performance and energy-efficiency for iso-accuracy across each. Overall, reducing the precision allows custom-designed compute units and lower buffering requirements to provide significant improvement in throughput.

Lowering precision (to 4-bits, 2-bits and even binary) and widening the filter banks gives as accurate network as those obtained with FP32 weights and activations.

Represent the short summary for finding the corresponding scientific article (long)

In this paper we introduced an algorithm to learn the connectivity of deep multi-branch networks. The approach is evaluated on image categorization where it consistently yields accuracy gains over state-of-the-art models that use fixed connectivity.