Question: Represent the scientific article to find a short TLDR

Question: In this paper, we are interested in two seemingly different concepts: \textit{adversarial training} and \textit{generative adversarial networks (GANs)}. Particularly, how these techniques work to improve each other. To this end, we analyze the limitation of adversarial training as a defense method, starting from questioning how well the robustness of a model can generalize. Then, we successfully improve the generalizability via data augmentation by the ``fake'' images sampled from generative adversarial network. After that, we are surprised to see that the ing robust classifier leads to a better generator, for free. We intuitively explain this interesting phenomenon and leave the theoretical analysis for future work. Motivated by these observations, we propose a system that combines generator, discriminator, and adversarial attacker together in a single network. After end-to-end training and fine tuning, our method can simultaneously improve the robustness of classifiers, measured by accuracy under strong adversarial attacks, and the quality of generators, evaluated both aesthetically and quantitatively. In terms of the classifier, we achieve better robustness than the state-of-the-art adversarial training algorithm proposed in (Madry \textit{et al.}, 2017), while our generator achieves competitive performance compared with SN-GAN . Deep neural networks have been very successful in modeling images, texts, and audios. Nonetheless, their characters have not yet been fully understood BID36, leaving a big hole for malicious attack algorithms. In this paper, we start from adversarial attacks and defense but try to find the connection with Generative Adversarial Network (GAN) BID10. Superficially, the difference between them is that the adversarial attack is the algorithm that finds a highly resembled image to cheat the classifier, whereas the GAN algorithm at its core is a generative model where the generator learns to convert white noise to images that look like authentic to the discriminator. We show in this paper that they are indeed closely related and can be used to strengthen each other: to accelerate and stabilize the GAN training cycle, the discriminator is expected to stay robust to adversarial examples; at the same time, a well trained generator provides a continuous support in probability space and thus improves the generalization ability of discriminator, even under adversarial attacks. That is the starting point of our idea to associate generative networks with robust classifiers. We find a novel way to make a connection between GAN and adversarial training. More importantly, we develop a system called AdvGAN to combine generator, discriminator, and adversarial attacker in the same network. Through the proposed "co-training" and "fine-tunning" steps, we are able to simultaneously improve the quality of generated images and the accuracy of discriminator under strong adversarial attacks. For example, when applying state-of-the-art adversarial training technique BID25, the accuracy of ResNet18(+CIFAR10) drops from 81.5% to 29.6%; whereas the accuracy of our discriminator network drops from 81.1% to 36.4% (keeping all the hyperparameters and network structure unchanged). For the generator side, we are able to match or even beat the inception score of state-of-the-art method on medium scale datasets (see Sec. 4 for details), with significantly fewer iterations. Lastly, we modify the loss of AC-GAN and our experiments confirm the superiority over the original one. Notations Throughout this paper, we denote the (image, label) pair as (x i, y i), i is the index of data point; The classifier parameterized by weights w is f (x; w), this function includes the final Softmax layer so the output is probabilities. We also define D(x) and G(z) as the discriminator and generator networks respectively. The adversarial example x adv is crafted by perturbing the original input, i.e. x adv = x + δ, where δ ≤ δ max. For convenience, we consider ∞ -norm in our experiments. The real and fake images are denoted as x real/fake, readers should differentiate the "fake" images with "adversarial" images 1. The training set is denoted as P real, this is the empirical distribution. Given the training set P real, we define empirical loss function DISPLAYFORM0 Generative adversarial network. This is a kind of algorithm that learns to model distribution either with or without supervision BID10, which is often considered as a hard task especially for high dimensional data (images, texts, audios, etc.). In recent years, GANs keep to be intensively studied, toghther with other competitive generative models such as variational autoencoder or VAE, which learns the latent representation of data via prior knowledge BID20, and auto-regressive model that models the conditional distribution given previous states (e.g. PixelCNN (van den) ). One advantage of GANs over other methods is that they are able to generate high quality images directly from certain distributions, whereas the other methods are either slow in generation, or yield blurry images. A GAN has two competing networks with different objectives: in the training phase, the generator G(z) and the discriminator D(x) are evolved in a minimax game, which can be denoted as a unified loss: min DISPLAYFORM0 Unlike traditional machine learning problems where we typically minimize the loss, is hard to optimize and that is the focus of recent literature. Among them, a guideline for the architectures of G and D is summarized in BID32. Other training techniques, including feature matching (similar to MMD-GAN BID23 BID1) and mini-batch discrimination are proposed in BID12 to improve the stability and quality of networks. For high resolution and photo-realistic image generation, currently the standard way is to first learn to generate low resolution images as the intermediate products, and then learn to refine them progressively BID8 BID19, this turns out to be more stable than directly generate high resolution images through a gigantic network. To reach the equilibrium efficiently, alternative loss metrics BID0 BID4 BID13 BID38 are applied and proven to be effective. Among them, BID0 theoretically explains why training the DCGAN is highly unstable -since the image manifold is highly concentrated towards a low dimensional manifold, and if two distributions P real and P fake are supported on two low dimensional manifolds that do not perfectly align, then there exists an "optimal discriminator D(x)" that tells apart two distributions with probability one. Moreover, under that situation, the gradient of discriminator ∇D(x) closes to zero and thus the training process is halted. Closely following that theorem, proposes to use Wasserstein-1 distance to measure the distance between real and fake data distribution. The ing network, namely "Wasserstein-GAN", largely improves the stability of GAN training. Another noteworthy work inspired by WGAN/WGAN-GP is spectral normalization, the idea is to estimate the operator norm σ max (W) of weights W inside layers (convolution, linear, etc.), and then normalize these weights to have 1-operator norm through dividing weight tensors by operator norm: W = W/σ max (W). Because ReLU non-linearity is already 1-Lipschitz, if we stack these layers together the network as a whole would still be 1-Lipschitz, that is exactly the prerequisite to apply Kantorovich-Rubinstein duality to estimate Wasserstein distance. Despite the success of aforementioned works, we want to address one missing part of these models: to the best of our knowledge, none of them consider the robustness of discrimination network D(x). This overlooked aspect can be problematic especially for high resolution images and large networks, this will be one of the central points of this paper. Adversarial attacks and defenses: Apart from GAN, another key ingredient of our method is adversarial examples, originated in BID36 and further studied in BID11. They found that machine learning models can be easily "fooled" by slightly modified images if we design a tiny perturbation according to some "attack" algorithms. In this paper we apply a simple yet efficient algorithm, namely PGD-attack BID25, to generate adversarial examples. Given an example x with ground truth label y, PGD computes adversarial perturbation δ by solving the following optimization with Projected Gradient Descent: DISPLAYFORM1 where f (·; w) is the network parameterized by weights w, (·, ·) is the loss function and for convenience we choose · to be the ∞ -norm in accordance with BID25, but note that other norms are also applicable. Intuitively, the idea of FORMULA2 is to find the point x adv:= x + δ within an ∞ -ball such that the loss value of x adv is maximized, so that point is most likely to be an adversarial example. In fact, most optimization based attacking algorithms (e.g. FGSM BID11, C&W BID6) shares the same idea as PGD attack. Opposite to the adversarial attacks, the adversarial defenses are techniques that make models resistant to adversarial examples. It is worth noting that defense is a much harder task compared with attacks, especially for high dimensional data combined with complex models. Despite that huge amount of defense methods are proposed BID31 BID25 BID5 BID24 BID14 BID9 BID40 BID33, many of them rely on gradient masking or obfuscation, which provide an "illusion" of safety. They claimed that the most effective defense algorithm is adversarial training BID25, formulated as DISPLAYFORM2 where (x, y) ∼ P real is the (image, label) joint distribution of real data, f (x; w) is the network parameterized by w, f (x; w), y is the loss function of network (such as the cross-entropy loss). We remark that the data distribution P real is often not available in practice, which will be replaced by the empirical distribution.3 PROPOSED APPROACH In Sec. 2 we listed some of the published works on adversarial defense, and pointed out that adversarial training is the most effective method to date. However, until now this method has only been tested on small dataset like MNIST and CIFAR10 and it is an open problem as to whether it scales to large dataset such as ImageNet. To our knowledge, there are two significant drawbacks of this method that restrict its application. First and most obviously, the overhead to find adversarial examples in each iteration is about 10x of the normal process (this can be inferred by #Iterations in each PGD attack FORMULA1 is obtained by adversarial training on CIFAR-10, we set δ max = 0.03125 in. The horizontal axis is the attack strength δ which is equivalent to δ max in. Note that δ max in FORMULA2 and FORMULA3 have different meaningsone is for attack and the other is for defense. Notice the increasing accuracy gap when δ < 0.03125. Right: The local Lipschitz value (LLV) measured by gradient norm ∂ ∂xi f (x i ; w), y i 2, data pairs (x i, y i) are chosen from the training and testing set respectively. During the training process, LLV on the training set stabilizes at a low level, whereas LLV on the test set keeps growing. methods such as. In essence, restricting the LLV can be formulated as a composite loss minimization problem: DISPLAYFORM0 Notice that can be regarded as the "one-step approximation" of. In practice we need to change the expectation over P real to empirical distribution of finite data, DISPLAYFORM1 where DISPLAYFORM2 are feature-label pairs constitute the training set. Ideally, if we have enough data and model size is moderate then the objective function in still converges to. However in practice when taking adversarial examples into account, we have one more problem to worry about: Does small LLV in training set imply small LLV in test set? The enlarged accuracy gap shown in FIG0 (Left) tends to give a negative answer. To verify this phenomenon directly, we calculate the LLV on images sampled from training and testing set respectively (FIG0), we observe that in parallel with accuracy gap, the LLV gap between training and testing set is equally significant. Thus we conclude that although adversarial training controls LLV around training set effectively, this property does not generalize to test set. Notice that our empirical finding does not contradict the certified robustness of adversarial training using generalization theory (e.g. BID34), which only explains weak attack situation. The generalization gap can be potentially reduced if we have a better understanding of P real instead of approximating it by training set. This leads to our first motivation: can we use GAN to learn P real and plug it into adversarial training algorithm to improve robustness on test set? We will give a possible solution in Sec. 3.3. GANs are notoriously hard to train. To our knowledge, there are two major symptoms of a failure trial -gradient vanishing and mode collapse. The theoretical explanation of gradient vanishing problem is discussed in BID0 by assuming the images lie in a low dimensional manifold. Following this idea, BID12 propose to use 1-Wasserstein distance in place of the KL-divergence. The central character of WGAN and improved WGAN is that they require the set of discriminators {D(x; w)|∀w ∈ R d } equals to the set of all 1-Lipschitz functions w.r.t input x. Practically, we can either clip the weight of discriminator w, or add a gradient norm regularizer BID12. Recently, another regularization technique called "spectral normalization" ) is proposed to enforce 1-Lipschitz discriminator and for the first time, GAN learns to generate high quality images from full ImageNet data with only one generator-discriminator pair. In contrast, AC-GAN BID30 -the supervised version of DCGAN -divides 1000 classes into 100 groups so each network-pair only learns 10 classes. Despite the success along this line of research, we wonder if a weaker assumption to the discriminator is possible. Concretely, instead of a strict one-Lipschitz function, we require a small local Lipschitz value on image manifold. Indeed, we find a connection between robustness of discriminator and the learning efficiency of generator, as illustrated in Fig. 2. Fake images Figure 2: Comparing robust and non-robust discriminators, for simplicity, we put them together into one graph. Conceptually, the non-robust discriminator tends to make all images close to the decision boundary, so even a tiny distortion δ can make a fake image x 0 to be classified as a real image x adv = x 0 + δ. In contrast, such δ is expected to be much larger for robust discriminators. As one can see in Fig. 2, if a discriminator D(x) has small LLV (or |D (x)|), then we know DISPLAYFORM0 for a "reasonably" large δ. In other words, for robust discriminator, the perturbed fake image x adv = x 0 + δ is unlikely to be mistakenly classified as real image, unless δ is large. Compared with adversarial attacks, the attacker is now a generator G(z; w) parameterized by w ∈ R d instead of the gradient ascend algorithm. For making x 0 "looks like" a real image (x adv), we must update generator G(z; w) to G(z; w) and by assuming the Lipschitz continuity of G, DISPLAYFORM1 This indicates the movement of generator weights w − w is lower bounded by the distance of a fake image x 0 to the decision boundary, specifically we have w − w ≥ δ /L G. Furthermore, recall that a robust discriminator D(x) implies a larger δ, putting them together we know that improving the robustness of discriminator will lead to larger updates of the generator. In Sec. 4 we experimentally show that adversarial training not only speeds up the convergence to the equilibrium, but also obtains an excellent generator. But we leave the rigorous analysis for future works. Motivated by Sec. 3.1 and 3.2, we propose a system that combines generator, discriminator, and adversarial attacker into a single network. Our system consists of two stages, the first stage is an end-to-end GAN training: the generator feeds fake images to the discriminator; meanwhile real images sampled from training set are processed by PGD attacking algorithm before sending to the discriminator. After that the discriminator is learned to minimize both discrimination loss and classification loss (introduced below). In the next stage, the discriminator is refined by combining the fake and real images. The network structure is illustrated in Fig. 3. In what follows, we give more details about each component: Discriminator: The discriminator could have the standard architecture like AC-GAN. In each iteration, it discriminates real and fake images. When the ground truth labels are available, it also predicts the classes. In this paper, we only consider the label-conditioning GANs BID27 BID30, whose architectural differences are briefly overviewed in FIG3. Among them we simply choose AC-GAN, despite that SN-GAN (a combination of spectral normalization and projection discriminator) performs much better in their paper. The reason we choose the AC-GAN is that Step 1. Co-trainingStep 2. Fine-tuning Figure 3: Illustration of the training process. Step-1 is the standard GAN training, i.e. alternatively updating the G and D networks. The only difference is that whenever feeding the real images to the D network, we first run 5 steps of PGD attack, so the discriminator is trained with adversarial examples. Step-2 is a refining technique, aiming at improving prediction accuracy on the test set. SN-GAN discriminator relies on the ground truth labels and their adversarial loss is not designed to encourage high classification accuracy. But surprisingly, even though AC-GAN is beaten by SN-GAN by a large margin, after inserting the adversarial training module, the performance of AC-GAN matches or even surpasses the SN-GAN, due to the reason discussed in Sec. 3.2. We also changed the loss objective of AC-GAN. Recall that the original loss in BID30 defined by discrimination likelihood L S and classification likelihood L C: DISPLAYFORM0 where X real/fake are any real/fake images, S is the discriminator output, C is the classifier output. Based on, the goal of discriminator is to maximize L S + L C while generator aims at maximizing L C − L S. According to this definition, both G and D are optimized to increase L C: even if G(z) produces unrecognizable images, D(x) has to struggle to classify them (with high loss), in such case the corresponding gradient term ∇L C can contribute uninformative direction to the discriminator. To resolve this issue, we split L C as follows, DISPLAYFORM1 then discriminator maximizes L S + L C1 and generator maximizes L C2 − L S. The new objective functions ensure that discriminator only focuses on classifying the real images and discriminating real/fake images. Generator: Similar to the traditional GAN training, the generator is updated on a regular basis to mimic the distribution of real data. This is the key ingredient to improve the robustness of discriminators: as shown in Sec. 3.1, adversarial training performs well on training set but is vulnerable on test set. Intuitively, this is because during adversarial training, the network only "sees" adversarial examples residing in the δ max -ball of all training samples, whereas the rest images in the data manifold are undefended. Data augmentation is a natural way to resolve this issue, but traditional techniques BID21 BID15 BID37 BID41 BID17 rely largely on combinations of geometric transforms to the training images, in our case the support of the probability density function is still very small. Instead, our system uses images sampled from generator to provide a continuously supported p.d.f. for the adversarial training. Unlike traditional augmentation methods, if the equilibrium in is reached, then we can show that one desirable solution of would be P fake (z) dist.= P real, and therefore the robust classifier can be trained on the learned distribution. Fine-tuning the classifier: This step aims at improving the classification accuracy, based on the auxiliary classifier in the pretrained discriminator. This is crucial because in the GAN training stage (step 1 in Fig. 3), the discriminator is not trained to minimize the classification error, but a weighted loss of both discrimination and classification. But in step 2, we want to focus on the robust classification task DISPLAYFORM2 where x adv = arg min DISPLAYFORM3 Here the function f (x; w) is just the classifier part of network D(x), recall that we are dealing with conditional GAN. As we can see, throughout the fine-tuning stage, we force the discriminator to focus on the classification task rather than the discrimination task. It turns out that the fine-tuning step boosts the accuracy by a large margin. Adversarial attacker is omitted in Fig. 3 due to width limit. We experiment on both CIFAR10 and a subset of ImageNet data. Specifically, we extract classes y i such that y i ∈ np.arange from the original ImageNet data: recall in total there are 1000 classes in ImageNet data and we sampled 294 − 151 = 143 classes from them. We choose these datasets because 1) the current state-of-the-art GAN, SN-GAN, also worked on these datasets, and 2) the current state-of-the-art adversarial training method BID25 only scales to CIFAR dataset. For fair comparison, we copy all the network architectures for generators and discriminators from SN-GAN, other important factors, such as learning rate, optimization algorithms, #discriminator updates in each cycle, etc. are also kept the same. The only modification is that we discarded the feature projection layer and applied the auxiliary classifier (see FIG3). Please refer to the appendix or source code for more implementation details. In what follows, we check whether fine-tuning helps improving test set accuracy. To this end, we design a experiment that compares two set of models: in the first set, we directly extract the auxiliary classifiers from discriminators to classify images; in the next set, we apply fine-tuning strategy to the pretrained model as Fig. 3 illustrated. The can be found in FIG4, which supports our argument that fine-tuning is indeed useful for better prediction accuracy. Robustness of discriminator: comparing robustness with/ without data augmentation In this experiment, we would like to compare the robustness of discriminator networks with or without data augmentation technique discussed in Sec. 3.3. The robustness is measured by the prediction accuracy under adversarial attack. For networks without data augmentation, that would be equal to the state-of-the-art Madry's algorithm BID25. For attacking algorithm, we choose the widely used ∞ PGD attack BID25, but other gradient based attacks are expected to Table 1: Accuracy of our model under ∞ PGD-attack. Inside the parenthesis is the improvement over standard adversarial training defense BID25.yield the same . We set the ∞ perturbation to σ max ∈ np.arange(0, 0.1, 0.01) as defined in. Another minor detail is that we scale the images to [−1, 1] rather than usual. This is because generators always have a tanh output layer, so we need to do some adaptations accordingly. We exhibit the in Tab. 1, showing our method can improve the robustness of state-of-the-art defensive algorithm. Effect of split classification loss Here we show the effect of split classification loss described in, recall that if we apply the loss in then the ing model is AC-GAN. It is known that AC-GAN can easily lose modes in practice, i.e. the generator simply ignores the noise input z and produces fixed images according to the label y. This defect is observed in many previous works BID16 BID26 BID18. In this ablation experiment, we compare the generated images trained by two loss functions, the is shown in FIG5. Quality of generator and convergence speed In the last experiment, we compare the quality of generators trained in three datasets: CIFAR10, ImageNet subset (64px) and ImageNet subset (128px). Our baseline model is the SN-GAN, considering that, as far as we know, SN-GAN is the best GAN model capable of learning hundreds of classes. Note that SN-GAN can also learn the conditional distribution of the entire ImageNet data (1000 classes), unfortunately, we are not able to match this experiment due to time and hardware limit. To show that the adversarial training technique indeed accelerates the convergence speed, we also tried to exclude adversarial training -this is basically an AC-GAN, except that an improved loss function discussed in is applied to discriminator D(x). The are exhibited in FIG6, which shows that adversarial training can improve the performance of GAN, and our generator achieves better inception score than SNGAN. Another finding is that our new loss proposed in FORMULA10 We compare the inception scores between our model and the SN-GAN. Clearly our method learns a high quality generator in a short time, specifically, in both datasets, AC-GAN with adversarial training surpasses SN-GAN in just 25 epochs (64px) or 50 epochs (128px). Another observation is that with adversarial training, the convergence is greatly accelerated.whether adversarial training with fake data augmentation really shrinks the generalization gap. To this end, we draw the same figure as FIG0, except that now the classification model is the discriminator after fine-tuning step (shown in Fig. 3). We compare the accuracy gap in FIG7. Clearly the model trained with the adversarial real+fake augmentation strategy works extremely well: it improves the testing accuracy under PGD-attack and so the generalization gap between training/testing set does not increase that much. In this paper, we draw a connection between adversarial training BID25 and generative adversarial network BID10. Our primary goal is to improve the generalization ability of adversarial training and this is achieved by data augmentation by the unlimited fake images. Independently, we see an improvement of both robustness and convergence speed in GAN training. While the theoretical principle in behind is still unclear to us, we gave an intuitive explanation. Apart from that, a minor contribution of our paper is the improved loss function of AC-GAN, showing a better in image quality.

Represent the short summary for finding the corresponding scientific article (long)

We found adversarial training not only speeds up the GAN training but also increases the image quality

Represent the TLDR for finding the corresponding scientific article:


Question: Represent the article for finding a single-sentence TLDR

Question: We present a method to train self-binarizing neural networks, that is, networks that evolve their weights and activations during training to become binary. To obtain similar binary networks, existing methods rely on the sign activation function. This function, however, has no gradients for non-zero values, which makes standard backpropagation impossible. To circumvent the difficulty of training a network relying on the sign activation function, these methods alternate between floating-point and binary representations of the network during training, which is sub-optimal and inefficient. We approach the binarization task by training on a unique representation involving a smooth activation function, which is iteratively sharpened during training until it becomes a binary representation equivalent to the sign activation function. Additionally, we introduce a new technique to perform binary batch normalization that simplifies the conventional batch normalization by transforming it into a simple comparison operation. This is unlike existing methods, which are forced to the retain the conventional floating-point-based batch normalization. Our binary networks, apart from displaying advantages of lower memory and computation as compared to conventional floating-point and binary networks, also show higher classification accuracy than existing state-of-the-art methods on multiple benchmark datasets. Deep learning has brought about remarkable advancements to the state-of-the-art in several fields including computer vision and natural language processing. In particular, convolutional neural networks (CNN's) have shown state-of-the-art performance in several tasks such as object recognition with AlexNet BID19, VGG BID29, ResNet and detection with R-CNN BID10 BID9 BID26. However, to achieve real-time performance these networks are dependent on specialized hardware like GPU's because they are computation and memory demanding. For example, AlexNet takes up 250Mb for its 60M parameters while VGG requires 528Mb for its 138M parameters. While the performance of deep networks has been gradually improving over the last few years, their computational speed has been steadily decreasing BID32. Notwithstanding this, interest has grown significantly in the deployment of CNN's in virtual reality headsets (Oculus, GearVR), augmented reality gear (HoloLens, Epson Moverio), and other wearable, mobile, and embedded devices. While such devices typically have very restricted power and memory capacites, they demand low latency and real-time performance to be able to provide a good user experience. Not surprisingly, there is considerable interest in making deep learning models computationally efficient to better suit such devices BID24 BID4 BID40.Several methods of compression, quantization, and dimensionality reduction have been introduced to lower memory and computation requirements. These methods produce near state-of-the-art , either with fewer parameters or with lower precision parameters, which is possible thanks to the redundancies in deep networks BID3.In this paper we focus on the solution involving binarization of weights and activations, which is the most extreme form of quantization. Binarized neural networks achieve high memory and computational efficiency while keeping performance comparable to their floating point counterparts. BID5 have shown that binary networks allow the replacement of multiplication and additions by simple bit-wise operations, which are both time and power efficient. The challenge in training a binary neural network is to convert all its parameters from the continuous domain to a binary representation, typically done using the sign activation function. However, since the gradient of sign is zero for all nonzero inputs, it makes standard back-propagation impossible. Existing state-of-the-art methods for network binarization BID5 BID25 alternate between a binarized forward pass and a floating point backward pass to circumvent this problem. In their case, the gradients for the sign activation are approximated during the backward pass, thus introducing inaccuracies in training. Furthermore, batch normalization BID16 is necessary in binary networks to avoid exploding feature map values due to the large scale of the weights. However, during inference, using batch normalization introduces intermediary floating point representations. This means, despite binarizing weights and activations, these networks can not be used on chips that do not support floating-point computations. In our method, the scaled hyperbolic tangent function tanh is used to bound the values in the range [−1, 1]. The network starts with floating point values for weights and activations, and progressively evolves into a binary network as the scaling factor is increased. Firstly, this means that we do not have to toggle between the binary and floating point weight representations. Secondly, we have a continuously differentiable function that allows backpropagation passes. As another important contribution, we reduce the standard batch normalization operation during the inference stage to a simple comparison. This modification is not only very efficient and can be accomplished using fixedpoint operations, it is also an order of magnitude faster than the floating-point counterpart. More clearly, while existing binarization methods perform, at each layer, the steps of binary convolutions, floating-point batch normalization, and sign activation, we only need to perform binary convolutions followed by our comparison-based batch normalization, which serves as the sign activation at the same time (see Fig. 1). convolutions, batch norm, sign activation convolutions, combined comparison batch norm and sign activation Figure 1: Schematic comparison of a typical binary neural network generated by existing methods (top half) with that of our method (bottom half). Existing methods are unable to avoid the 32-bit floating point batch norm, while we simplify it to a comparison operation, which simultaneously serves as the activation layer. We validate the performance of our self-binarizing networks by comparing them to those of existing binarization methods. We choose the standard bechmarks of CIFAR-10, CIFAR-100 BID18 ) as well as ImageNet BID27 and popular network architectures such as VGG and AlexNet. We demonstrate higher accuracies despite using less memory and fewer computations. To the best of our knowledge, our proposed networks are the only ones that are free of any floating point computations and can therefore be deployed on low-precision integrated chips or micro-controllers. In what follows, in Sec. 2 we describe previous work related to reducing the network complexity and where we are placed among them. In Sec. 3 we explain how the scaled tanh function can be used for progressive binarization. In Sec. 4 we explain our binarization method for weights and activations and explain how to simplify batch normalization at inference time. In Sec. 5 we compare our technique to existing state-of-the-art binary networks on standard benchmarks and demonstrate the performance gains from our proposed batch normalization simplification. Sec. 6 concludes the paper. Making deep networks memory and computation efficient has been approached in various ways. In this section we cover some of the relevant literature and explain how our work is positioned with respect to the state-of-the-art. The interested reader may refer to BID4, BID24, and for a wider coverage. Since most of the computation in deep networks is due to convolutions, it is logical to focus on reducing the computational burden due to them. Howard et al. FORMULA0; BID34 BID8 employ variations of convolutional layers by taking advantage of the separability of the kernels either by convolving with a group of input channels or by splitting the kernels across the dimensions. In addition to depth-wise convolutions, MobileNetV2 BID28 uses inverted residual structures to learn how to combine the inputs in a residual block. In general, these methods try to design a model that computes convolutions in an efficient manner. This is different from this work because it focuses on redesigning the structure of the network, while ours tries to reduce the memory requirements by using lower precision parameters. However, our method can be applied to these networks as well. Reducing the number of weights likewise reduces the computational and memory burden. Due to the redundancy in deep neural networks BID3, there are some weights that contribute more to the output than others. The process of removing the less contributing weights is called pruning. In BID20, the contribution of weights is measured by the effect on the training error when this parameter is zeroed. In Deep Compression BID11, the weights with lowest absolute value are pruned and the remaining are quantized and compressed using Huffman coding. In other work, Fisher Information BID31 ) is used to measure the amount of information the output carries about each parameter, which allows pruning. While these approaches often operate on already trained networks and fine-tune them after compression, our method trains a network to a binary state from scratch without removing any parameters or feature maps. This allows us to retain the original structure of the network, while still leaving the potential for further compression after or during binarization using pruning techniques. Another way to reduce memory consumption and potentially improve computational efficiency is the quantization of weights. Quantized neural networks BID36 occupy less memory while retaining similar performance as their full precision counterparts. DoReFaNet BID36 proposes to train low bitwidth networks with low bitwidth gradients using stochastic quantization. Similarly, devise a weight partitioning technique to incrementally quantize the network at each step. The degree of quantization can vary between techniques. BID7, BID37, and BID22 quantize weights to three levels, i.e., two bits only. These quantizations, while severe, still allow for accurate inference. However, to improve computational efficency, specialized hardware is needed to take advantage of the underlying ternary operations. An extreme form of such quantization is binarization, which requires only one bit to represent. Expectation BackPropagation (EBP) paved the way for training neural networks for precision limitedhardware BID30. BinaryConnect BID5 extended the idea to train neural networks with binary weights. The authors later propose BinaryNet to binarize activations as well. Similarly, XNORNet BID25 extends BinaryNet by adding a scaling factor to the parameters of every layer while keeping the weights and activations binary. ABCNet approximates full precision parameters as a linear combination of binary weights and uses multiple binary activations to compensate for the information loss arising from quantization. BID13 use Hessian approximations to minimize loss with respect to the binary weights during training. The focus of our work is the binarization of weights and activations of a network. In previous binarization methods, the binarization process is non-differentiable leading to approximations during the training that can affect the final accuracy. In contrast, we use a differentiable function to pro-gressively self-binarize the network and improve its accuracy. Additionally, we differ from these techniques as we introduce a comparison-based binary batch normalization that eliminates all floating point operations at inference time. In typical binary network training, during the forward pass, the floating-point weights and activations are quantized to binary values {−1, 1}, through a piece-wise constant function, most commonly the sign function: DISPLAYFORM0 This non-linear activation leads to strong artifacts during the forward pass, and does not generate gradients for backpropagation. The derivatives of the binarized weights are therefore approximately computed using a Straight Through Estimator (STE) BID1. STE creates non-zero derivative approximations for functions that either have a zero derivative everywhere or are nondifferentiable. Typically, the derivative of sign is estimated by using the following STE: DISPLAYFORM1 In the backward propagation step, the gradients are computed on the binarized weights using the STE and the corresponding floating-point representations are updated. Since both forward and backward functions are different, the training is ill-defined. The lack of an accurate derivative for the weights and activations creates a mismatch between the quantized and floating-point values and influences learning accuracy. This kind of problems has been studied previously, and continuation methods have been proposed to simplify its solution BID0. To do so, the original complex and non-smooth optimization problem is transformed by smoothing the original function and then gradually decreasing the smoothness during training, building a sequence of sub-optimization problems converging to the original one. For example, BID2 apply these methods on the last layer of a neural network to predict hashes from images. Following this philosophy, we introduce a much simpler and efficient training method that allows the network to self-binarize. We pass all our weights and activations through the hyperbolic tangent function tanh whose slope can be varied using a scale parameter ν > 0. As seen in FIG0, when ν is large enough the tanh(νx) converges to the sign(x) function: DISPLAYFORM2 Throughout the training process, the weights and activations use floating-point values. Starting from a value of ν = 1, as the scale factor is progressively incremented, the weights and activations are forced to attain binary values {−1, 1}. During the training and while ν is still small, the derivative of tanh exists for every value of x and is expressed as DISPLAYFORM3 where sech is the hyperbolic secant. Using the scaled tanh, we can build a continuously differentiable network which progressively approaches a binary state, leading to a more principled approach to obtain a binarized network. In this section, we formally describe our self-binarizing approach. We first explain how weights and activations are binarized. Then, we propose a more efficient, comparison-based batch-normalization method that is more suitable when working in binary settings. As stated above, we cannot use binary weights at training time as it would make gradient computation infeasible. Instead, we use a set of constrained floating-point weights W at each layer. Unlike traditional networks, these weights are not learnable parameters of our model, but depend on learnable parameters. For each layer of the network, we define a set of learnable, unconstrained parameters P and use the scaled tanh to compute the weights as DISPLAYFORM0 where ν e is the scale factor at epoch e, taken from a sequence 1 = ν 0 < ν 1 <... < ν M → ∞ of increasingly larger values. During training, parameters P are updated to minimize a loss function using a standard gradient-based optimization procedure, such as stochastic gradient descent. The scaled tanh transforms the parameters P to obtain weights W that are bounded in the range [−1, 1] and become closer to binary values as the training proceeds. At the end of the training, weights W are very close to exact binary values. At this point, we obtain the final binary weights B by taking the sign of the parameters, DISPLAYFORM1 At inference time, we drop all learnable parameters P and constrained weights W, and use only binary weights B. Just as for weights, we cannot use binary activations either at training time. We follow the idea as above to address activation self-binarization as well. During training, we use the scaled tanh as the activation function of the network. For a given layer, the activation function transforms the output O of the layer to lead to the activation DISPLAYFORM0 The activations are constrained to lie within the range [−1, 1], and eventually become binary at the end of the training procedure. At inference time we make the network completely binary by substituting the scaled tanh by the sign operator as the binary activation function. Batch Normalization (BN) introduced by BID16 accelerates the training of a general deep network. During training, the BN layers compute the running mean µ r and standard deviation σ r of the feature maps that pass through them, as well as two parameters, β and γ, that define an affine transformation. Later, at inference time, BN layers normalize and transform the input I to obtain an output O as given by DISPLAYFORM0 For binary networks BID5 BID25 in particular, BN becomes essential in order to avoid exploding activation values. However, using BN brings in the limitation of relying on floating-point computations. Apart from affecting computation and memory requirements, the floating-point BN effectively eliminates the possibility of using the network on low-precision hardware. A useful observation of BN is that, in a networks with binary activations, the output of the BN layers is always fed into a sign function. This means only the sign of the normalized output is kept, while its magnitude is not relevant for the subsequent layers. We can leverage this property to simplify the BN operation. The sign of the output O of a BN layer can be reformulated as: DISPLAYFORM1 with DISPLAYFORM2 While T is a floating-point value, in practice we represent it as a fixed-point 8-bit integer. This sacrifices some amount of numerical precision, but we observed no negative effect on the performance of our models. We refer to our simplified batch normalization as Binary Batch Normalization (BinaryBN).Note that the derivation of Eq. FORMULA8 does not hold when γ = 0. This is handled as a special case that simply evaluates β > 0. It must be emphasized that BinaryBN is not an approximate method; it computes the exact value of the sign of the output of the standard BN.During training we use the conventional BN layers. At inference time, we replace them with the BinaryBN without any loss in prediction accuracy. Our trained models can thus bypass the floatingpoint batch normalization with an efficient alternative that requires mere comparison operations. In this section, we first compare our self-binarization method against other known techniques. Later on, we discuss and demonstrate the efficiency gained using our proposed BinaryBN instead of the typical BN layer. We compare our self-binarizing networks with other state-of-the-art methods that use binary networks. BID5 present BinaryConnect (BC), a method that only binarizes the weights. Binary Neural Networks (BNN) improves BC by also binarizing the activations. Similarly, BID25 present two variants of their method: Binary Weight Networks (BWN), which only binarizes the weights, and XNORnet (XNOR), which binarizes both weights and activations. For a fair comparison and to prove the generality of our method, we use the original implementations of BC, BNN, BWN, and XNOR, and apply our self-binarizing technique to them. Specifically, we substituted the weights from the original implementations by a pair of parameters P and constrained weights W given by Eq.. Also, we substituted the activation functions by the scaled tanh as described in Eq. FORMULA6. At inference time we used the binary weights obtained with Eq. and the sign operator as the activation function. Additionally, we replace the batch normalization layers by our BinaryBN layer for the cases where the activations are binarized. We evaluate the methods on three common benchmark datasets: CIFAR-10, CIFAR-100 BID18 ) and ILSVRC12 ImageNet BID27. For CIFAR-10 and CIFAR-100, we use a VGG-16-like network with data augmentation as proposed in BID21: 4 pixels are padded on each side, and a 32x32 patch is randomly cropped from the padded image or its horizontal flip. During testing, only a single view of the original 32x32 image is evaluated. The model is trained with a mini-batch size of 256 for 100 epochs. The ILSVRC12 ImageNet dataset is used to train an AlexNet-like network without drop-out or local response normalization layers. We use the data augmentation strategy from. At inference time, only the center crop is used from the validation set. The model is trained with a mini-batch size of 64 and a total of 50 epochs. In all our experiments we increase ν from 1 to 1000 during training in an exponential manner. The final ν = 1000 is large enough to make weights and activations almost binary in practice. We optimize all models using Adam BID17 with an exponentially decaying learning rate starting at 10 −3. TAB0 shows the of the experiments. Our self-binarizing approach achieves the highest accuracies for CIFAR-10 and ImageNet. Our method is only slightly outperformed by BC for CIFAR-100, but still gives better Top-5 accuracy for the same model. For both weights and activation binarization, our method obtains the best across all datasets and architectures. What is remarkable is that the improved performance comes despite eliminating floating-point computations and using drastically fewer bits than the previous methods, as can be seen in the columns B w, B a and B BN of TAB0. For CIFAR-10 and CIFAR-100, BWN outperforms the full precision models likely because the binarization serves as a regularizer BID5. With all other computations being common between our self-binarizing networks and other networks with binary weights and activations, any difference in computational efficiency has to arise from the use of a different batch normalization scheme. We therefore compare our proposed BinaryBN layer to the conventional BN layer as well as to the Shift-based Batch Normalization (SBN) proposed by. SBN proposes to round both γ and σ r to their nearest power of 2 and replace multiplications and divisions by left and right shifts, respectively. SBN follows the same formula as BN, and is used both at training and inference time, so that the network is trained with the rounded parameters. Table. 2 summarizes the requirements of memory and computational time of these three types of batch normalization layers. We assume the standard case where a binary convolution is followed by a batch normalization layer and then a binary activation function. For storing BN parameters, we need four 32-bit vectors of length c, amounting to 32×4c = 128c bits, c being the number of channels in this layer. For SBN, we need two 32-bit vectors and two 8-bit vectors of length c, ing in 80c bits. For BinaryBN, we need an 8-bit vector of size c to store the T value of each channel, and a binary vector for the sign of γ, totaling 9c bits. We experimentally assessed the time and memory requirements of the three batch normalization techniques. We run batches of increasing sizes through BN, SBN and BinaryBN layers with randomly generated values for µ r, σ r, β, and γ, and measure time and memory consumption. FIG1 shows the . Overall, BinaryBN is nearly one order of magnitude less memory consuming and faster than BN and SBN. We present a novel method to binarize a deep network that is principled, simple, and in binarization of weights and activations. Instead of relying on the sign function, we use the tanh function with a controllable slope. This simplifies the training process without breaking the flow of derivatives in the back-propagation phase as compared to that of existing methods that have to toggle between floating-point and binary representations. In addition to this, we replace the conventional batch normalization, which forces existing binarization methods to use floating point computations, by a simpler comparison operation that is directly adapted to networks with binary activations. Our simplified batch normalization is not only computationally trivial, it is also extremely memoryefficient. Our trained binary networks outperform those of existing binarization schemes on standard benchmarks despite using lesser memory and computation.

Represent the TLDR for finding the corresponding scientific article

A method to binarize both weights and activations of a deep neural network that is efficient in computation and memory usage and performs better than the state-of-the-art.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We present a new approach, SNIP, that is simple, versatile and interpretable; it prunes irrelevant connections for a given task at single-shot prior to training and is applicable to a variety of neural network models without modifications.


Question: Represent the following document

Question: In many applications, the training data for a machine learning task is partitioned across multiple nodes, and aggregating this data may be infeasible due to storage, communication, or privacy constraints. In this work, we present Good-Enough Model Spaces (GEMS), a novel framework for learning a global satisficing (i.e. "good-enough") model within a few communication rounds by carefully combining the space of local nodes' satisficing models. In experiments on benchmark and medical datasets, our approach outperforms other baseline aggregation techniques such as ensembling or model averaging, and performs comparably to the ideal non-distributed models. There has been significant work in designing distributed optimization methods in response to challenges arising from a wide range of large-scale learning applications. These methods typically aim to train a global model by performing numerous communication rounds between distributed nodes. However, most approaches treat communication reduction as an objective, not a constraint, and seek to minimize the number of communication rounds while maintaining model performance. Less explored is the inverse setting-where our communication budget is fixed and we aim to maximize accuracy while restricting communication to only a few rounds. These few-shot model aggregation methods are ideal when any of the following conditions holds:• Limited network infrastructure: Distributed optimization methods typically require a connected network to support the collection of numerous learning updates. Such a network can be difficult to set up and maintain, especially in settings where devices may represent different organizational entities (e.g., a network of different hospitals).• Privacy and data ephemerality: Privacy policies or regulations like GDPR may require nodes to periodically delete the raw local data. Few-shot methods enable learning an aggregate model in ephemeral settings, where a node may lose access to its raw data. Additionally, as fewer messages are sent between nodes, these methods have the potential to offer increased privacy benefits.• Extreme asynchronicity: Even in settings where privacy is not a concern, messages from distributed nodes may be unevenly spaced and sporadically communicated over days, weeks, or even months (e.g., in the case of remote sensor networks or satellites). Few-shot methods drastically limit communication and thus reduce the wall-clock time required to learn an aggregate model. Throughout this paper, we reference a simple motivating example. Consider two hospitals, A and B, which each maintain private (unshareable) patient data pertinent to some disease. As A and B are geographically distant, the patients they serve sometimes exhibit different symptoms. Without sharing the raw training data, A and B would like to jointly learn a single model capable of generalizing to a wide range of patients. The prevalent learning paradigm in this settingdistributed or federated optimization-dictates that A and B share iterative model updates (e.g., gradient information) over a network. From a meta-learning or multitask perspective, we can view each hospital (node) as a separate learning task, where our goal is to learn a single aggregate model which performs well on each task. However, these schemes often make similar assumptions on aggregating data and learning updates from different tasks. As a promising alternative, we present good-enough model spaces (GEMS), a framework for learning an aggregate model over distributed nodes within a small number of communication rounds. Intuitively, the key idea in GEMS is to take advantage of the fact that many possible hypotheses may yield'good enough' performance for a learning task on local data, and that considering the intersection between these sets can allow us to compute a global model quickly and easily. Our proposed approach has several advantages. First, it is simple and interpretable in that each node only communicates its locally optimal model and a small amount of metadata corresponding to local performance. Second, each node's message scales linearly in the local model size. Finally, GEMS is modular, allowing the operator to tradeoff the aggregate model's size against its performance via a hyperparameter.We make the following contributions in this work. First, we present a general formulation of the GEMS framework. Second, we offer a method for calculating the good-enough space on each node as a R d ball. We empirically validate GEMS on both standard benchmarks (MNIST and CIFAR-10) as well as a domain-specific health dataset. We consider learning convex classifiers and neural networks in standard distributed setups as well as scenarios in which some small global held-out data may be used for fine-tuning. We find that on average, GEMS increases the accuracy of local baselines by 10.1 points and comes within 43% of the (unachievable) global ideal. With fine-tuning, GEMS increases the accuracy of local baselines by 41.3 points and comes within 86% of the global ideal. Distributed Learning. Current distributed and federated learning approaches typically rely on iterative optimization techniques to learn a global model, continually communicating updates between nodes until convergence is reached. To improve the overall runtime, a key goal in most distributed learning methods is to minimize communication for some fixed model performance; to this end, numerous methods have been proposed for communication-efficient and asynchronous distributed optimization (e.g., ; ; ; ; ; Richtárik & Takáč, 2016; ;). In this work, our goal is instead to maximize performance for a fixed communication budget (e.g., only one or possibly a few rounds of communication).One-shot/Few-shot Methods. While simple one-shot distributed communication schemes, such as model averaging, have been explored in convex settings (; ; ; ;), guarantees typically rely on data being partitioned in an IID manner and over a small number of nodes relative to the total number of samples. Averaging can also perform arbitrarily poorly in non-convex settings, particularly when the local models converge to differing local optima . Other one-shot schemes leverage ensemble methods, where an ensemble is constructed from models trained on distinct partitions of the data (; ;). While these ensembles can often yield good performance in terms of accuracy, a concern is that the ing ensemble size can become quite large. In Section 4, we compare against these one-shot baselines empirically, and find in that GEMs can outperform both simple averaging and ensembles methods while requiring significantly fewer parameters. Meta-learning and transfer learning. The goals of metalearning and transfer learning are seemingly related, as these works aim to share knowledge from one learning process onto others. However, in the case of transfer learning, methods are typically concerned with one-way transfer-i.e., optimizing the performance of a single target model, not jointly aggregate knowledge between multiple models. In meta-learning, such joint optimization is performed, but similar to traditional distributed optimization methods, it is assumed that these models can be updated in an iterative fashion, with potentially numerous rounds of communication being performed throughout the training process. Version Spaces. In developing GEMS, we draw inspiration from work in version space learning, an approach for characterizing the set of logical hypotheses consistent with available data . Similar to , we observe that if each node communicates its version space to the central server, the server can return a consistent hypothesis in the intersection of all node version spaces. However, assume that the hypotheses of interest are consistent with the observed data-i.e., they perfectly predict the correct outcomes. Our approach significantly generalizes to explore imperfect, noisy hypotheses spaces as more commonly observed in practice. As in traditional distributed learning, we assume a training set DISPLAYFORM0..} as the subset of training examples belonging to node k, such that DISPLAYFORM1 We assume that a single node (e.g., a central server) can aggregate updates communicated in the network. Fixing a function class H, our goal is to learn an aggregate model h G ∈ H that approximates the performance of the optimal model h * ∈ H over S while limiting communication to one (or possibly a few) rounds of communication. In developing a method for model aggregation, our intuition is that the aggregate model should be at least good-enough over each node's local data, i.e., it should achieve some minimum performance for the task at hand. Thus, we can compute h G by having each node compute and communicate a set of locally good-enough models to a central server, which learns h G from the intersection of these sets. DISPLAYFORM2 denote a model evaluation function, which determines whether a given model h is good-enough over a sample of data points {(x i, y i)} d ⊆ S. In this work, define "good-enough" in terms of the accuracy of h and a threshold: DISPLAYFORM3 Using these model evaluation functions, we formalize the proposed approach for model aggregation, GEMS, in Algorithm 1. In GEMS, each node k = 1,..., K computes the set of models DISPLAYFORM4 and sends it to the central node. After collecting H 1,...H K, the central node selects h G from the intersection of the sets, ∩ i H i. When granted access to a small sample of public data, the server can additionally use this auxiliary data further fine-tune the selected h ∈ ∩ i H i, an approach we discuss further below. FIG3 visualizes this approach for a model class with only two weights (w 1 and w 2) and two learners ("red" and "blue"). The'good-enough' model space, H k, for each learner is a set of regions over the weight space (the blue regions correspond to one learner and the red regions correspond to second learner). The final aggregate model, h G, is selected from the area in which the spaces intersect. For a fixed hypothesis class H, applying Algorithm 1 requires two components: (i) a mechanism for computing H k over every node, and (ii) a mechanism for identifying the aggregate model, h G ∈ ∩ k H k. In this work, we present methods for two types of models: convex models and simple neural networks. For convex models, we find that H k can be approximated as R d -ball in the parameter space, requiring only a single round of communication between nodes to learn h G. For neural networks, we apply Algorithm 1 to each layer in a step-wise fashion, compute H k as a set of independent R d -balls corresponding to every neuron in the layer, and identify intersections between different neurons. This requires one round of communication per layer (a few rounds for the entire network).We can compute these R d balls by fixing the center at the optimal local model on a device. The radius for the ball is computed via binary search: at each iteration, the node samples a candidate hypothesis h and evaluates Q(h, S k). The goal is to identify that largest radius such that all models located in the R d ball are good-enough. Algorithm 2 presents a simple method for constructing H k. More details can be found in Appendix A (convex setting) and Appendix B (neural network setting). DISPLAYFORM5 Node k computes good-enough model space, H k, according to 4: end for DISPLAYFORM6 else 10:Set R upper = R end if 12: end while 13: Return H k Fine-tuning. In many contexts, a small sample of public data S public may be available to the central server. This could correspond to a public research dataset, or devices which have waived their privacy right. The coordinating server can fine-tune H G on S public by updating the weights for a small number of epochs. We find that fine-tuning is particularly useful for improving the quality of the GEMS aggregate model, H G, compared to other baselines. We now present the evaluation for GEMS on three datasets: MNIST , CIFAR-10 , and HAM10000 , a medical imaging dataset. HAM10000 (HAM) consists of images of skin lesions, and our model is tasked with distinguishing between 7 types of lesions. Full details can be found in Appendix C.1. We focus on the performance of GEMS for neural networks, and discuss for convex models in Appendix A. We partitioned data by label, such that all train/validation images corresponding to a particular label would be assigned to the same node. We consider three baselines: 1) global, a model trained on data aggregated across all nodes, 2) local, the average performance of models trained locally on each node, and 3) naive average, a parameter-wise average of all local models. All are reported on the aggregated test set consisting of all test data across all nodes. Fine-tuning consists of updating the last layer's weights of the GEMS model for 5 epochs over a random sample of 1000 images from the aggregated validation data. We report the average accuracy (and standard deviation) of all over 5 trials. Neural network performance. We evaluated the neural network variant of GEMS on simple two layer feedforward neural networks TAB0 ). The precise network configuration and training details are outlined in Appendix C.4. In the majority of cases, the untuned GEMS model outperforms the local/average baselines. Moreover, fine-tuning has a significant impact, and tuned GEMS model 1) significantly outperforms every baseline, and 2) does not degrade as K increases. In Appendix F, we demonstrate that GEMS is more parameter efficient than ensemble baselines, delivering better accuracy with fewer parameters. Fine-tuning. The in TAB0 suggest that fine-tuning on a holdout set of samples S public has a significant effect on the GEMS model. We evaluate the effect of fine-tuning as the number of public data samples (the size of the tuning set) changes. For neural networks FIG1 ), finetuned GEMS consistently outperforms 1) the finetuned baselines, and 2) a'raw' model trained directly on S public. This suggest that the GEMS model is learning weights that are more amenable to fine-tuning, and are perhaps capturing better representations for the overall task. Though this advantage diminishes as the tuning sample size increases, the advantage of GEMS is especially pronounced for smaller samples, and achieves remarkable improvements with just 100 images. Intersection Analysis. In certain cases, GEMS may not find an intersection between different nodes. This occurs when the task is too complex for the model, or is set too high. In practice, we notice that finding an intersection requires us to be conservative (e.g low values) when setting for each node. We explain this by our choice to represent H k as an R d ball. Though R d balls are easy to compute and intersect, they're fairly coarse approximations of the actual good-enough model space. To illustrate node behavior at different settings of, we defer the reader to experiments performed in Appendix G. In summary, we introduce GEMS, a framework for learning an aggregated model across different nodes within a few rounds of communication. We validate one approach for constructing good-enough model spaces (as R d balls) on three datasets for both convex classifiers and simple feedforward networks. Despite the simplicity of the proposed approach, we find that it outperforms a wide range of baselines for effective model aggregation TAB0 We provide a more detailed explanation of the GEMS algorithm for convex settings. Consider the class of linear separators f w (·) parameterized by a weight vector w ∈ R d. For each node k, we compute H k as R d -ball in the parameter space, represented as a tuple (c k ∈ R d, r k ∈ R) corresponding to the center and radius. Formally, H k = {w ∈ R d |||c k − w|| 2 ≤ r k}. Fixing as our minimum acceptable performance, we want to compute H k such that ∀w ∈ H k, Q(w, S k) = 1. Intuitively, every model contained within the d-ball should have an accuracy greater than or equal to. DISPLAYFORM0 over node data S k, is fixed hyperparameter, and Q(·) is a minimum accuracy threshold defined according to Eq. 1. R max and ∆ define the scope and stopping criteria for the binary search. Intersection: Given K nodes with individual DISPLAYFORM1., K}. We pick a point in this intersection by solving: DISPLAYFORM2 which takes a minimum value of 0 when w ∈ ∩ i H i. This w can be improved by fine-tuning on a limited sample of'public data'. We provide a more detailed explanation of the GEMS algorithm applied to neural networks. First, we observe that the final layer of an MLP is a linear model. Hence, we can apply the method above with no modification. However, the input to this final layer is a set of stacked, non-linear transformations which extract feature from the data. For these layers, the approach presented above faces two challenges:1. Node specific features: When the distribution of data is non-i.i.d across nodes, different nodes may learn different feature extractors in lower layers.2. Model Isomorphisms: MLPs are extremely sensitive to the weight initialization. Two models trained on the same set of samples (with different initializations) may have equivalent behavior despite learning weights. In particular, reordering a model's hidden neurons (within the same layer) does not alter the model's predictions, but corresponds to a different weight vector w. In order to construct H k for hidden layers, we modify the approach presented in Appendix A, applying it to individual hidden units. Formally, let the ordered set DISPLAYFORM0 Broadly, Q neuron returns 1 if the output of f w over z j−1 is within of z j l, and −1 otherwise. We can now apply Algorithm 2 to each neuron. Formally:1. Each node k learns a locally optimal model m k, with optimal neuron weights w j l *, over all j, l.2. Fix hidden layer j = 1. Apply Algorithm 2 to each DISPLAYFORM1, with Q(·) according to Eq 3 and predefined hyperparameter j. Denote the R d ball constructed for neuron l as H k j,l. to the central server which constructs the aggregate hidden layer f Gj,· such ∀i, k, ∃i: f Gj,i ∈ H k j,i. This is achieved by greedily applying Eq 2 to tuples in the cartesian product H 1 j,· ×... × H K j,·. Neurons for which no intersection exists are included in f Gj,·, thus trivially ensuring the condition above.4. The server sends h Gj,· to each node, which insert h Gj,· at layer j in their local models and retrain the layers above j.5. Increment j, and return to if any hidden layers remain. Step FORMULA10 is expensive for large L and K as |H DISPLAYFORM0 A simplifying assumption is that if H k j,i and H k j,l are'far', then the likelihood of intersection is low. Operationalizing this, we can perform k-means clustering over all neurons. In step, we now only look for intersections between tuples of neurons in the same cluster. Neurons for which no intersection exists are included in f Gj,·. For notational clarity, we denote the number of clusters with which k-means is run as m, in order to distinguish it from device index k. We describe preprocessing/featurization steps for our empirical . MNIST. We used the standard MNIST dataset. CIFAR-10. We featurize CIFAR-10 (train, test, and validation sets) using a pretrained ImageNet VGG-16 model from Keras. All models are learned on these featurized images. HAM10000. The HAM dataset consists of 10015 images of skin lesions. Lesions are classified as one of seven potential types: actinic keratoses and intraepithelial carcinoma (akiec), basal cell carcinoma (bcc), benign keratosis (bkl), dermatofibroma (df), melanoma (mel), melanocytic nevi (nv), and vascular lesions (vasc). As FIG4 shows, the original original dataset is highly skewed, with almost 66% of images belonging to one class. In order to balance the dataset, we augment each class by performing a series of random transformations (rotations, width shifts, height shifts, vertical flips, and horizontal flips) via Keras . We sample 2000 images from each class. We initially experimented with extracting ImageNet features (similar to our proceedure for CIFAR-10). However, training a model on these extractions ed in poor performance. We constructed our own feature extractor, by training a simple convolutional network on 66% of the data, and trimming the final 2 dense layers. This network contained 3 convolutional layers (32, 64, 128 filters with 3 × 3 kernels) interspersed with 2 × 2 MaxPool layers, and followed by a single hidden layer with 512 neurons. Given K nodes, we partitioned each dataset in order to ensure that all images corresponding to the same class belonged to the same node. TAB3 provides an explicit breakdown of the label partitions for each of the three datasets, across the different values of K we experimented with. We divided each dataset into train, validation, and test splits. All training occurs exclusively on the train split and all are reported for performance on the test split. We use the validation split to construct each node's Our convex model consists of a simple logistic regression classifier. We train with Adam, a learning rate of 0.001, and a batch size of 32. We terminate training when training accuracy converges. Our non-convex model consists of a simple two layer feedforward neural network. For MNIST and HAM, we fix the hidden layer size to 50 neurons. For CIFAR-10, we fix the hidden layer size to 100 neurons. We apply dropout with a rate of 0.5 to the hidden layer. We train with Adam, a learning rate of 0.001, and a batch size of 32. We terminate training when training accuracy converges. We evaluate the convex variant of GEMS on logistic classifiers. The for all three datasets for a varying number nodes K is presented in TAB6. Fine-tuning consists of updating the weights of the GEMS model for 5 epochs over a random sample of 1000 images from the aggregated validation data. Training details are provided in Appendix C.3In a convex setting, we find that GEMS frequently defaults to a weighted average of the parameters. Hence, the GEMS closely mirror naive averaging. As the number of agents increases, both untuned GEMS and the baselines significantly decrease in performance. However, tuned GEMS remains relatively consistent, and outperforms all other baselines. We use = 0.70 for MNIST, = 0.40 for HAM, and = 0.20 for CIFAR-10. E. Neural Network Results Table 4 presents the neural network for MNIST. We use = 0.7 for the final layer, and let j denote the deviation allowed for the hidden neurons (as defined in Eq 3). For neural networks, GEMS provides a modular framework to tradeoff between the model size and performance, via hyperparameters m (the number of clusters created when identifying intersections) and j (the maximum output deviation allowed for hidden neurons). Intuitively both parameters control the number of hidden neurons in the aggregate model h G. Table 7 compares adjustments for j and m on CIFAR-10 for 5 nodes against an ensemble of local device models. We observe that the GEMS performance correlates with the number of hidden neurons, and that GEMS outperforms the ensemble method at all settings (despite having fewer parameters). For ease of clarity, we describe the model size in terms of the number of hidden neurons. For ensembles, we sum the hidden neurons across all ensemble members. All are averaged over 5 trials, with standard deviations reported. We notice that in order for GEMs to find an intersection, we have to set conservatively. We illustrate this phenomenon in FIG8. We consider the convex MNIST case (K = 2), and do a grid search over different values of for each node. We plot whether an intersection was identified, and the ing accuracy at that setting TAB0 Model Aggregation via Good-Enough Model Spaces TAB0 Model Aggregation via Good-Enough Model Spaces

We present Good-Enough Model Spaces (GEMS), a framework for learning an aggregate model over distributed nodes within a small number of communication rounds.

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.