Question: Represent the scientific article to find a short TLDR

Question: We propose the Wasserstein Auto-Encoder (WAE)---a new algorithm for building a generative model of the data distribution. WAE minimizes a penalized form of the Wasserstein distance between the model distribution and the target distribution, which leads to a different regularizer than the one used by the Variational Auto-Encoder (VAE). This regularizer encourages the encoded training distribution to match the prior. We compare our algorithm with several other techniques and show that it is a generalization of adversarial auto-encoders (AAE). Our experiments show that WAE shares many of the properties of VAEs (stable training, encoder-decoder architecture, nice latent manifold structure) while generating samples of better quality. The field of representation learning was initially driven by supervised approaches, with impressive using large labelled datasets. Unsupervised generative modeling, in contrast, used to be a domain governed by probabilistic approaches focusing on low-dimensional data. Recent years have seen a convergence of those two approaches. In the new field that formed at the intersection, variational auto-encoders (VAEs) BID16 constitute one well-established approach, theoretically elegant yet with the drawback that they tend to generate blurry samples when applied to natural images. In contrast, generative adversarial networks (GANs) BID9 turned out to be more impressive in terms of the visual quality of images sampled from the model, but come without an encoder, have been reported harder to train, and suffer from the "mode collapse" problem where the ing model is unable to capture all the variability in the true data distribution. There has been a flurry of activity in assaying numerous configurations of GANs as well as combinations of VAEs and GANs. A unifying framework combining the best of GANs and VAEs in a principled way is yet to be discovered. This work builds up on the theoretical analysis presented in BID3. Following; BID3, we approach generative modeling from the optimal transport (OT) point of view. The OT cost is a way to measure a distance between probability distributions and provides a much weaker topology than many others, including f -divergences associated with the original GAN algorithms BID25. This is particularly important in applications, where data is usually supported on low dimensional manifolds in the input space X. As a , stronger notions of distances (such as f -divergences, which capture the density ratio between distributions) often max out, providing no useful gradients for training. In contrast, OT was claimed to have a nicer behaviour BID11 although it requires, in its GAN-like implementation, the addition of a constraint or a regularization term into the objective.: Both VAE and WAE minimize two terms: the reconstruction cost and the regularizer penalizing discrepancy between P Z and distribution induced by the encoder Q. VAE forces Q(Z|X = x) to match P Z for all the different input examples x drawn from P X. This is illustrated on picture (a), where every single red ball is forced to match P Z depicted as the white shape. Red balls start intersecting, which leads to problems with reconstruction. In contrast, WAE forces the continuous mixture Q Z:= Q(Z|X)dP X to match P Z, as depicted with the green ball in picture (b). As a latent codes of different examples get a chance to stay far away from each other, promoting a better reconstruction. In this work we aim at minimizing OT W c (P X, P G) between the true (but unknown) data distribution P X and a latent variable model P G specified by the prior distribution P Z of latent codes Z ∈ Z and the generative model P G (X|Z) of the data points X ∈ X given Z. Our main contributions are listed below (cf. also FIG0):• A new family of regularized auto-encoders (Algorithms 1, 2 and Eq. 4), which we call Wasserstein Auto-Encoders (WAE), that minimize the optimal transport W c (P X, P G) for any cost function c. Similarly to VAE, the objective of WAE is composed of two terms: the c-reconstruction cost and a regularizer D Z (P Z, Q Z) penalizing a discrepancy between two distributions in Z: P Z and a distribution of encoded data points, i.e. DISPLAYFORM0 When c is the squared cost and D Z is the GAN objective, WAE coincides with adversarial auto-encoders of BID23.• Empirical evaluation of WAE on MNIST and CelebA datasets with squared cost c(x, y) = x − y 2 2. Our experiments show that WAE keeps the good properties of VAEs (stable training, encoder-decoder architecture, and a nice latent manifold structure) while generating samples of better quality, approaching those of GANs.• We propose and examine two different regularizers D Z (P Z, Q Z). One is based on GANs and adversarial training in the latent space Z. The other uses the maximum mean discrepancy, which is known to perform well when matching high-dimensional standard normal distributions P Z BID10. Importantly, the second option leads to a fully adversary-free min-min optimization problem.• Finally, the theoretical considerations presented in BID3 and used here to derive the WAE objective might be interesting in their own right. In particular, Theorem 1 shows that in the case of generative models, the primal form of W c (P X, P G) is equivalent to a problem involving the optimization of a probabilistic encoder Q(Z|X).The paper is structured as follows. In Section 2 we review a novel auto-encoder formulation for OT between P X and the latent variable model P G derived in BID3. Relaxing the ing constrained optimization problem we arrive at an objective of Wasserstein auto-encoders. We propose two different regularizers, leading to WAE-GAN and WAE-MMD algorithms. Section 3 discusses the related work. We present the experimental in Section 4 and conclude by pointing out some promising directions for future work. Our new method minimizes the optimal transport cost W c (P X, P G) based on the novel auto-encoder formulation (see Theorem 1 below). In the ing optimization problem the decoder tries to accurately reconstruct the encoded training examples as measured by the cost function c. The encoder tries to simultaneously achieve two conflicting goals: it tries to match the encoded distribution of training examples Q Z:= E P X [Q(Z|X)] to the prior P Z as measured by any specified divergence D Z (Q Z, P Z), while making sure that the latent codes provided to the decoder are informative enough to reconstruct the encoded training examples. This is schematically depicted on FIG0. We use calligraphic letters (i.e. X) for sets, capital letters (i.e. X) for random variables, and lower case letters (i.e. x) for their values. We denote probability distributions with capital letters (i.e. P (X)) and corresponding densities with lower case letters (i.e. p(x)). In this work we will consider several measures of discrepancy between probability distributions P X and P G. The class of f -divergences BID21 ) is defined by DISPLAYFORM0 A rich class of divergences between probability distributions is induced by the optimal transport (OT) problem . Kantorovich's formulation of the problem is given by DISPLAYFORM0 where c(x, y): X × X → R + is any measurable cost function and P(X ∼ P X, Y ∼ P G) is a set of all joint distributions of (X, Y) with marginals P X and P G respectively. A particularly interesting case is when (X, d) is a metric space and c(x, y) = d p (x, y) for p ≥ 1. In this case W p, the p-th root of W c, is called the p-Wasserstein distance. When c(x, y) = d(x, y) the following Kantorovich-Rubinstein duality holds 1: DISPLAYFORM1 where F L is the class of all bounded 1-Lipschitz functions on (X, d). One way to look at modern generative models like VAEs and GANs is to postulate that they are trying to minimize certain discrepancy measures between the data distribution P X and the model P G. Unfortunately, most of the standard divergences known in the literature, including those listed above, are hard or even impossible to compute, especially when P X is unknown and P G is parametrized by deep neural networks. Previous research provides several tricks to address this issue. In case of minimizing the KL-divergence D KL (P X, P G), or equivalently maximizing the marginal log-likelihood E P X [log p G (X)], the famous variational lower bound provides a theoretically grounded framework successfully employed by VAEs BID16 BID24. More generally, if the goal is to minimize the f -divergence D f (P X, P G) (with one example being D KL), one can resort to its dual formulation and make use of f -GANs and the adversarial training BID25. Finally, OT cost W c (P X, P G) is yet another option, which can be, thanks to the celebrated Kantorovich-Rubinstein duality, expressed as an adversarial objective as implemented by the Wasserstein-GAN. We include an extended review of all these methods in Supplementary A.In this work we will focus on latent variable models P G defined by a two-step procedure, where first a code Z is sampled from a fixed distribution P Z on a latent space Z and then Z is mapped to the image X ∈ X = R d with a (possibly random) transformation. This in a density of the form DISPLAYFORM0 assuming all involved densities are properly defined. For simplicity we will focus on non-random decoders, i.e. generative models P G (X|Z) deterministically mapping Z to X = G(Z) for a given map G: Z → X. Similar for random decoders can be found in Supplementary B.1.It turns out that under this model, the OT cost takes a simpler form as the transportation plan factors through the map G: instead of finding a coupling Γ in between two random variables living in the X space, one distributed according to P X and the other one according to P G, it is sufficient to find a conditional distribution DISPLAYFORM1 is identical to the prior distribution P Z. This is the content of the theorem below proved in BID3. To make this paper self contained we repeat the proof in Supplementary B.Theorem 1 For P G as defined above with deterministic P G (X|Z) and any function G: DISPLAYFORM2 where Q Z is the marginal distribution of Z when X ∼ P X and Z ∼ Q(Z|X).This allows us to optimize over random encoders Q(Z|X) instead of optimizing over all couplings between X and Y. Of course, both problems are still constrained. In order to implement a numerical solution we relax the constraints on Q Z by adding a penalty to the objective. This finally leads us to the WAE objective: DISPLAYFORM3 where Q is any nonparametric set of probabilistic encoders, D Z is an arbitrary divergence between Q Z and P Z, and λ > 0 is a hyperparameter. Similarly to VAE, we propose to use deep neural networks to parametrize both encoders Q and decoders G. Note that as opposed to VAEs, the WAE formulation allows for non-random encoders deterministically mapping inputs to their latent codes. We propose two different penalties D Z (Q Z, P Z): DISPLAYFORM4 and use the adversarial training to estimate it. Specifically, we introduce an adversary (discriminator) in the latent space Z trying to separate 2 "true" points sampled from P Z and "fake" ones sampled from Q Z BID9. This in the WAE-GAN described in Algorithm 1. Even though WAE-GAN falls back to the min-max problem, we move the adversary from the input (pixel) space X to the latent space Z. On top of that, P Z may have a nice shape with a single mode (for a Gaussian prior), in which case the task should be easier than matching an unknown, complex, and possibly multi-modal distributions as usually done in GANs. This is also a reason for our second penalty: DISPLAYFORM5 For a positive-definite reproducing kernel k: Z × Z → R the following expression is called the maximum mean discrepancy (MMD): DISPLAYFORM6 where H k is the RKHS of real-valued functions mapping Z to R. If k is characteristic then MMD k defines a metric and can be used as a divergence measure. We propose to use DISPLAYFORM7 Fortunately, MMD has an unbiased U-statistic estimator, which can be used in conjunction with stochastic gradient descent (SGD) methods. This in the WAE-MMD described in Algorithm 2. It is well known that the maximum mean discrepancy performs well when matching high-dimensional standard normal distributions BID10 ) so we expect this penalty to work especially well working with the Gaussian prior P Z. Require: Regularization coefficient λ > 0. Initialize the parameters of the encoder Q φ, decoder G θ, and latent discriminator Dγ. while (φ, θ) not converged do Sample {x1, . . ., xn} from the training set Sample {z1, . . ., zn} from the prior PZ Samplezi from Q φ (Z|xi) for i = 1,..., n Update Dγ by ascending: DISPLAYFORM0 Update Q φ and G θ by descending: DISPLAYFORM1 end while ALGORITHM 2 Wasserstein Auto-Encoder with MMD-based penalty (WAE-MMD).Require: Regularization coefficient λ > 0, characteristic positive-definite kernel k. Initialize the parameters of the encoder Q φ, decoder G θ, and latent discriminator Dγ. while (φ, θ) not converged do Sample {x1, . . ., xn} from the training set Sample {z1, . . ., zn} from the prior PZ Samplezi from Q φ (Z|xi) for i = 1,..., n Update Q φ and G θ by descending: DISPLAYFORM2 end whileWe point out once again that the encoders Q φ (Z|x) in Algorithms 1 and 2 can be non-random, i.e. deterministically mapping input points to the latent codes. In this case Q φ (Z|x) = δ µ φ (x) for a function µ φ: X → Z and in order to samplez i from Q φ (Z|x i) we just need to return µ φ (x i). Literature on auto-encoders Classical unregularized auto-encoders minimize only the reconstruction cost. This in different training points being encoded into non-overlapping zones chaotically scattered all across the Z space with "holes" in between where the decoder mapping P G (X|Z) has never been trained. Overall, the encoder Q(Z|X) trained in this way does not provide a useful representation and sampling from the latent space Z becomes hard BID1.Variational auto-encoders BID16 ) minimize a variational bound on the KLdivergence D KL (P X, P G) which is composed of the reconstruction cost plus the regularizer DISPLAYFORM0 The regularizer captures how distinct the image by the encoder of each training example is from the prior P Z, which is not guaranteeing that the overall encoded distribution E P X [Q(Z|X)] matches P Z like WAE does. Also, VAEs require non-degenerate (i.e. nondeterministic) Gaussian encoders and random decoders for which the term log p G (x|z) can be computed and differentiated with respect to the parameters. Later BID24 proposed a way to use VAE with non-Gaussian encoders. WAE minimizes the optimal transport W c (P X, P G) and allows both probabilistic and deterministic encoder-decoder pairs of any kind. The VAE regularizer can be also equivalently written BID13 as a sum of D KL (Q Z, P Z) and a mutual information I Q (X, Z) between the images X and latent codes Z jointly distributed according to P X × Q(Z|X). This observation provides another intuitive way to explain a difference between our algorithm and VAEs: WAEs simply drop the mutual information term I Q (X, Z) in the VAE regularizer. When used with c(x, y) = x − y 2 2 WAE-GAN is equivalent to adversarial auto-encoders (AAE) proposed by BID23. Theory of BID3 (and in particular Theorem 1) thus suggests that AAEs minimize the 2-Wasserstein distance between P X and P G. This provides the first theoretical justification for AAEs known to the authors. WAE generalizes AAE in two ways: first, it can use any cost function c in the input space X; second, it can use any discrepancy measure D Z in the latent space Z (for instance MMD), not necessarily the adversarial one of WAE-GAN.Finally, Zhao et al. (2017b) independently proposed a regularized auto-encoder objective similar to BID3 and our based on very different motivations and arguments. Following VAEs their objective (called InfoVAE) defines the reconstruction cost in the image space implicitly through the negative log likelihood term − log p G (x|z), which should be properly normalized for all z ∈ Z. In theory VAE and InfoVAE can both induce arbitrary cost functions, however in practice this may require an estimation of the normalizing constant (partition function) which can 3 be different for different values of z. WAEs specify the cost c(x, y) explicitly and don't constrain it in any way. Literature on address computing the OT cost in large scale using SGD and sampling. They approach this task either through the dual formulation, or via a regularized version of the primal. They do not discuss any implications for generative modeling. Our approach is based on the primal form of OT, we arrive at regularizers which are very different, and our main focus is on generative modeling. The WGAN minimizes the 1-Wasserstein distance W 1 (P X, P G) for generative modeling. The authors approach this task from the dual form. Their algorithm comes without an encoder and can not be readily applied to any other cost W c, because the neat form of the Kantorovich-Rubinstein duality holds only for W 1. WAE approaches the same problem from the primal form, can be applied for any cost function c, and comes naturally with an encoder. In order to compute the values or of OT we need to handle non-trivial constraints, either on the coupling distribution Γ or on the function f being considered. Various approaches have been proposed in the literature to circumvent this difficulty. For W 1 tried to implement the constraint in the dual formulation by clipping the weights of the neural network f. Later BID11 proposed to relax the same constraint by penalizing the objective of with a term λ · E (∇f (X) − 1) 2 which should not be greater than 1 if f ∈ F L. In a more general OT setting of W c BID5 proposed to penalize the objective of with the KLdivergence λ · D KL (Γ, P ⊗ Q) between the coupling distribution and the product of marginals. BID8 showed that this entropic regularization drops the constraints on functions in the dual formulation as opposed to. Finally, in the context of unbalanced optimal transport it has been proposed to relax the constraint in by regularizing the objective with BID4 BID20, where Γ X and Γ Y are marginals of Γ. In this paper we propose to relax OT in a way similar to the unbalanced optimal transport, i.e. by adding additional divergences to the objective. However, we show that in the particular context of generative modeling, only one extra divergence is necessary. DISPLAYFORM1 Literature on GANs Many of the GAN variations (including f -GAN and WGAN) come without an encoder. Often it may be desirable to reconstruct the latent codes and use the learned manifold, in which cases these models are not applicable. There have been many other approaches trying to blend the adversarial training of GANs with autoencoder architectures (a; BID6 BID29 BID2 . The approach proposed by BID29 is perhaps the most relevant to our work. The authors use the discrepancy between Q Z and the distribution E Z ∼P Z [Q Z|G(Z) ] of auto-encoded noise vectors as the objective for the max-min game between the encoder and decoder respectively. While the authors showed that the saddle points correspond to P X = P G, they admit that encoders and decoders trained in this way have no incentive to be reciprocal. As a workaround they propose to include an additional reconstruction term to the objective. WAE does not necessarily lead to a min-max game, uses a different penalty, and has a clear theoretical foundation. Several works used reproducing kernels in context of GANs. BID19; BID7 use MMD with a fixed kernel k to match P X and P G directly in the input space X. These methods have been criticised to require larger mini-batches during training: estimating MMD k (P X, P G) requires number of samples roughly proportional to the dimensionality of the input space X BID28 which is typically larger than 10 3. BID18 take a similar approach but further train k adversarially so as to arrive at a meaningful loss function. WAE-MMD uses MMD to match Q Z to the prior P Z in the latent space Z. Typically Z has no more than 100 dimensions and P Z is Gaussian, which allows us to use regular mini-batch sizes to accurately estimate MMD. Random samples In this section we empirically evaluate 4 the proposed WAE model. We would like to test if WAE can simultaneously achieve (i) accurate reconstructions of data points, (ii) reasonable geometry of the latent manifold, and (iii) random samples of good (visual) quality. Importantly, the model should generalize well: requirements (i) and (ii) should be met on both training and test data. We trained WAE-GAN and WAE-MMD (Algorithms 1 and 2) on two real-world datasets: MNIST consisting of 70k images and CelebA BID22 containing roughly 203k images. Experimental setup In all reported experiments we used Euclidian latent spaces Z = R dz for various d z depending on the complexity of the dataset, isotropic Gaussian prior distributions P Z (Z) = N (Z; 0, σ 2 z · I d) over Z, and a squared cost function c(x, y) = x − y 2 2 for data points x, y ∈ X = R dx. We used deterministic encoder-decoder pairs, Adam BID15 ) with β 1 = 0.5, β 2 = 0.999, and convolutional deep neural network architectures for encoder mapping µ φ: X → Z and decoder mapping G θ: Z → X similar to the DCGAN ones reported by BID27 with batch normalization BID14. We tried various values of λ and noticed that λ = 10 seems to work good across all datasets we considered. Since we are using deterministic encoders, choosing d z larger than intrinsic dimensionality of the dataset would force the encoded distribution Q Z to live on a manifold in Z. This would make matching Q Z to P Z impossible if P Z is Gaussian and may lead to numerical instabilities. We use d z = 8 for MNIST and d z = 64 for CelebA which seems to work reasonably well. Random samples VAE WAE-MMD WAE-GAN Figure 3: VAE (left column), WAE-MMD (middle column), and WAE-GAN (right column) trained on CelebA dataset. In "test reconstructions" odd rows correspond to the real test points. We also report of VAEs. VAEs used the same latent spaces as discussed above and standard Gaussian priors P Z = N (0, I d). We used Gaussian encoders Q(Z|X) = N Z; µ φ (X), Σ(X) with mean µ φ and diagonal covariance Σ. For both MNIST and CelebA we used Bernoulli decoders parametrized by G θ. Functions µ φ, Σ, and G θ were parametrized by deep nets of the same architectures as used in WAE. In WAE-GAN we used discriminator D composed of several fully connected layers with ReLu. We tried WAE-MMD with the RBF kernel but observed that it fails to penalize the outliers of Q Z because of the quick tail decay. If the codesz = µ φ (x) for some of the training points x ∈ X end up far away from the support of P Z (which may happen in the early stages of training) the corresponding terms in the U-statistic k(z,z) = e − z−z 2 2 /σ 2 k will quickly approach zero and provide no gradient for those outliers. This could be avoided by choosing the kernel bandwidth σ 2 k in a data-dependent manner, however in this case per-minibatch U-statistic would not provide an unbiased estimate for the gradient. Instead, we used the inverse multiquadratics kernel k(x, y) = C/(C + x − y 2 2) which is also characteristic and has much heavier tails. In all experiments we used C = 2d z σ 2 z, which is the expected squared distance between two multivariate Gaussian vectors drawn from P Z. This significantly improved the performance compared to the RBF kernel (even the one with σ Random samples are generated by sampling P Z and decoding the ing noise vectors z into G θ (z). As expected, in our experiments we observed that for both WAE-GAN and WAE-MMD the quality of samples strongly depends on how accurately Q Z matches P Z. To see this, notice that during training the decoder function G θ is presented only with encoded versions µ φ (X) of the data points X ∼ P X. Indeed, the decoder is trained on samples from Q Z and thus there is no reason to expect good when feeding it with samples from P Z. In our experiments we noticed that even slight differences between Q Z and P Z may affect the quality of samples. In some cases WAE-GAN seems to lead to a better matching and generates better samples than WAE-MMD. However, due to adversarial training WAE-GAN is highly unstable, while WAE-MMD has a very stable training much like VAE. In order to quantitatively assess the quality of the generated images, we use the Fréchet Inception Distance introduced by BID12 and report the on CelebA in Table 1. These confirm that the sampled images from WAE are of better quality than from VAE, and WAE-GAN gets a slightly better score than WAE-MMD, which correlates with visual inspection of the images. Test reconstructions and interpolations. We take random points x from the held out test set and report their auto-encoded versions G θ (µ φ (x)). Next, pairs (x, y) of different data points are sampled randomly from the held out test set and encoded: z x = µ φ (x), z y = µ φ (y). We linearly interpolate between z x and z y with equally-sized steps in the latent space and show decoded images. Using the optimal transport cost, we have derived Wasserstein auto-encoders-a new family of algorithms for building generative models. We discussed their relations to other probabilistic modeling techniques. We conducted experiments using two particular implementations of the proposed method, showing that in comparison to VAEs, the images sampled from the trained WAE models are of better quality, without compromising the stability of training and the quality of reconstruction. Future work will include further exploration of the criteria for matching the encoded distribution Q Z to the prior distribution P Z, assaying the possibility of adversarially training the cost function c in the input space X, and a theoretical analysis of the dual formulations for WAE-GAN and WAE-MMD. Even though GANs and VAEs are quite different-both in terms of the conceptual frameworks and empirical performance-they share important features: (a) both can be trained by sampling from the model P G without knowing an analytical form of its density and (b) both can be scaled up with SGD. As a , it becomes possible to use highly flexible implicit models P G defined by a twostep procedure, where first a code Z is sampled from a fixed distribution P Z on a latent space Z and then Z is mapped to the image G(Z) ∈ X = R d with a (possibly random) transformation G: Z → X. This in latent variable models P G of the form.These models are indeed easy to sample and, provided G can be differentiated analytically with respect to its parameters, P G can be trained with SGD. The field is growing rapidly and numerous variations of VAEs and GANs are available in the literature. Next we introduce and compare several of them. The original generative adversarial network (GAN) BID9 approach minimizes DISPLAYFORM0 with respect to a deterministic decoder G: Z → X, where T is any non-parametric class of choice. It is known that D GAN (P X, P G) ≤ 2 · D JS (P X, P G) − log and the inequality turns into identity in the nonparametric limit, that is when the class T becomes rich enough to represent all functions mapping X to. Hence, GANs are minimizing a lower bound on the JS-divergence. However, GANs are not only linked to the JS-divergence: the f -GAN approach BID25 showed that a slight modification D f,GAN of the objective allows to lower bound any desired f -divergence in a similar way. In practice, both decoder G and discriminator T are trained in alternating SGD steps. Stopping criteria as well as adequate evaluation of the trained GAN models remain open questions. Recently, the authors of argued that the 1-Wasserstein distance W 1, which is known to induce a much weaker topology than D JS, may be better suited for generative modeling. When P X and P G are supported on largely disjoint low-dimensional manifolds (which may be the case in applications), D KL, D JS, and other strong distances between P X and P G max out and no longer provide useful gradients for P G. This "vanishing gradient" problem necessitates complicated scheduling between the G/T updates. In contrast, W 1 is still sensible in these cases and provides stable gradients. The Wasserstein GAN (WGAN) minimizes DISPLAYFORM1 where W is any subset of 1-Lipschitz functions on X. It follows from that D WGAN (P X, P G) ≤ W 1 (P X, P G) and thus WGAN is minimizing a lower bound on the 1-Wasserstein distance. Variational auto-encoders (VAE) BID16 utilize models P G of the form FORMULA4 and minimize DISPLAYFORM2 with respect to a random decoder mapping P G (X|Z). The conditional distribution P G (X|Z) is often parametrized by a deep net G and can have any form as long as its density p G (x|z) can be computed and differentiated with respect to the parameters of G. A typical choice is to use Gaussians P G (X|Z) = N (X; G(Z), σ 2 · I). If Q is the set of all conditional probability distributions Q(Z|X), the objective of VAE coincides with the negative marginal log-likelihood D VAE (P X, P G) = −E P X [log P G (X)]. However, in order to make the D KL term of tractable in closed form, the original implementation of VAE uses a standard normal P Z and restricts Q to a class of Gaussian distributions Q(Z|X) = N Z; µ(X), Σ(X) with mean µ and diagonal covariance Σ parametrized by deep nets. As a consequence, VAE is minimizing an upper bound on the negative log-likelihood or, equivalently, on the KL-divergence D KL (P X, P G).One possible way to reduce the gap between the true negative log-likelihood and the upper bound provided by D VAE is to enlarge the class Q. Adversarial variational Bayes (AVB) BID24 follows this argument by employing the idea of GANs. Given any point x ∈ X, a noise ∼ N, and any fixed transformation e: X × R → Z, a random variable e(x,) implicitly defines one particular conditional distribution Q e (Z|X = x). AVB allows Q to contain all such distributions for different choices of e, replaces the intractable term D KL Q e (Z|X), P Z in BID31 by the adversarial approximation D f,GAN corresponding to the KL-divergence, and proposes to minimize DISPLAYFORM3 The D KL term in may be viewed as a regularizer. Indeed, VAE reduces to the classical unregularized auto-encoder if this term is dropped, minimizing the reconstruction cost of the encoder-decoder pair Q(Z|X), P G (X|Z). This often in different training points being encoded into nonoverlapping zones chaotically scattered all across the Z space with "holes" in between where the decoder mapping P G (X|Z) has never been trained. Overall, the encoder Q(Z|X) trained in this way does not provide a useful representation and sampling from the latent space Z becomes hard BID1.Adversarial auto-encoders (AAE) BID23 replace the D KL term in with another regularizer: DISPLAYFORM4 where Q Z is the marginal distribution of Z when first X is sampled from P X and then Z is sampled from Q(Z|X), also known as the aggregated posterior BID23. Similarly to AVB, there is no clear link to log-likelihood, as D AAE ≤ D AVB. The authors of BID23 argue that matching Q Z to P Z in this way ensures that there are no "holes" left in the latent space Z and P G (X|Z) generates reasonable samples whenever Z ∼ P Z. They also report an equally good performance of different types of conditional distributions Q(Z|X), including Gaussians as used in VAEs, implicit models Q e as used in AVB, and deterministic encoder mappings, i.e. Q(Z|X) = δ µ(X) with µ: X → Z. We will consider certain sets of joint probability distributions of three random variables (X, Y, Z) ∈ X × X × Z. The reader may wish to think of X as true images, Y as images sampled from the model, and Z as latent codes. We denote by P G,Z (Y, Z) a joint distribution of a variable pair (Y, Z), where Z is first sampled from P Z and next Y from P G (Y |Z). Note that P G defined in and used throughout this work is the marginal distribution of Y when (Y, Z) ∼ P G,Z.In the optimal transport problem, we consider joint distributions Γ(X, Y) which are called couplings between values of X and Y. Because of the marginal constraint, we can write Γ(X, Y) = Γ(Y |X)P X (X) and we can consider Γ(Y |X) as a non-deterministic mapping from X to Y. Theorem 1. shows how to factor this mapping through Z, i.e., decompose it into an encoding distribution Q(Z|X) and the generating distribution P G (Y |Z).As in Section 2.2, P(X ∼ P X, Y ∼ P G) denotes the set of all joint distributions of (X, Y) with marginals P X, P G, and likewise for P(X ∼ P X, Z ∼ P Z). The set of all joint distributions of (X, Y, Z) such that X ∼ P X, (Y, Z) ∼ P G,Z, and (Y ⊥ ⊥ X)|Z will be denoted by P X,Y,Z. Finally, we denote by P X,Y and P X,Z the sets of marginals on (X, Y) and (X, Z) (respectively) induced by distributions in P X,Y,Z. Note that P(P X, P G), P X,Y,Z, and P X,Y depend on the choice of conditional distributions P G (Y |Z), while P X,Z does not. In fact, it is easy to check that P X,Z = P(X ∼ P X, Z ∼ P Z). From the definitions it is clear that P X,Y ⊆ P(P X, P G) and we immediately get the following upper bound: DISPLAYFORM0 If P G (Y |Z) are Dirac measures (i.e., Y = G(Z)), it turns out that P X,Y = P(P X, P G):adversary in WAE-GAN to α = 5 × 10 −4. After 30 epochs we decreased both by factor of 2, and after first 50 epochs further by factor of 5.Both encoder and decoder used fully convolutional architectures with 4x4 convolutional filters. Encoder architecture: DISPLAYFORM1 Decoder architecture: DISPLAYFORM2 Adversary architecture for WAE-GAN: DISPLAYFORM3 Here Conv k stands for a convolution with k filters, FSConv k for the fractional strided convolution with k filters (first two of them were doubling the resolution, the third one kept it constant), BN for the batch normalization, ReLU for the rectified linear units, and FC k for the fully connected layer mapping to R k. All the convolutions in the encoder used vertical and horizontal strides 2 and SAME padding. Finally, we used two heuristics. First, we always pretrained separately the encoder for several minibatch steps before the main training stage so that the sample mean and covariance of Q Z would try to match those of P Z. Second, while training we were adding a pixel-wise Gaussian noise truncated at 0.01 to all the images before feeding them to the encoder, which was meant to make the encoders random. We played with all possible ways of combining these two heuristics and noticed that together they in slightly (almost negligibly) better compared to using only one or none of them. Our VAE model used cross-entropy loss (Bernoulli decoder) and otherwise same architectures and hyperparameters as listed above. We pre-processed CelebA images by first taking a 140x140 center crops and then resizing to the 64x64 resolution. We used mini-batches of size 100 and trained the models for various number of epochs (up to 250). All reported WAE models were trained for 55 epochs and VAE for 68 epochs. For WAE-MMD we used λ = 100 and for WAE-GAN λ = 1. Both used σ 2 z = 2. For WAE-MMD the learning rate of Adam was initially set to α = 10 −3. For WAE-GAN the learning rate of Adam for the encoder-decoder pair was initially set to α = 3 × 10 −4 and for the adversary to 10 −3. All learning rates were decreased by factor of 2 after 30 epochs, further by factor of 5 after 50 first epochs, and finally additional factor of 10 after 100 first epochs. Both encoder and decoder used fully convolutional architectures with 5x5 convolutional filters. Encoder architecture:x ∈ R 64×64×3 → Conv 128 → BN → ReLU For WAE-GAN we used a heuristic proposed in Supplementary IV of BID24. Notice that the theoretically optimal discriminator would in D * (z) = log p Z (z) − log q Z (z), where p Z and q Z are densities of P Z and Q Z respectively. In our experiments we added the log prior log p Z (z) explicitly to the adversary output as we know it analytically. This should hopefully make it easier for the adversary to learn the remaining Q Z density term. Our VAE model used a cross-entropy reconstruction loss (Bernoulli decoder) and α = 10 −4 as the initial Adam learning rate and the same decay schedule as explained above. Otherwise all the architectures and hyperparameters were as explained above.

We propose a new auto-encoder based on the Wasserstein distance, which improves on the sampling properties of VAE.

An empirical study of variational inference based on chi-square divergence minimization, showing that minimizing the CUBO is trickier than maximizing the ELBO


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: In this paper, we aim to develop a novel mechanism to preserve differential privacy (DP) in adversarial learning for deep neural networks, with provable robustness to adversarial examples. We leverage the sequential composition theory in DP, to establish a new connection between DP preservation and provable robustness. To address the trade-off among model utility, privacy loss, and robustness, we design an original, differentially private, adversarial objective function, based on the post-processing property in DP, to tighten the sensitivity of our model. An end-to-end theoretical analysis and thorough evaluations show that our mechanism notably improves the robustness of DP deep neural networks. The pervasiveness of machine learning exposes new vulnerabilities in software systems, in which deployed machine learning models can be used (a) to reveal sensitive information in private training data , and/or (b) to make the models misclassify, such as adversarial examples . Efforts to prevent such attacks typically seek one of three solutions: Models which preserve differential privacy (DP) , a rigorous formulation of privacy in probabilistic terms; Adversarial training algorithms, which augment training data to consist of benign examples and adversarial examples crafted during the training process, thereby empirically increasing the classification accuracy given adversarial examples ; and Provable robustness, in which the model classification given adversarial examples is theoretically guaranteed to be consistent, i.e., a small perturbation in the input does not change the predicted label . On the one hand, private models, trained with existing privacy-preserving mechanisms (; ; ; 2017b; a; ;), are unshielded under adversarial examples. On the other hand, robust models, trained with adversarial learning algorithms (with or without provable robustness to adversarial examples), do not offer privacy protections to the training data. That one-sided approach poses serious risks to machine learning-based systems; since adversaries can attack a deployed model by using both privacy inference attacks and adversarial examples. To be safe, a model must be i) private to protect the training data, and ii) robust to adversarial examples. Unfortunately, there has not yet been research on how to develop such a model, which thus remains a largely open challenge. Simply combining existing DP-preserving mechanisms and provable robustness conditions (; ;) cannot solve the problem, for many reasons. (a) Existing sensitivity bounds (; 2017b; a) and designs have not been developed to protect the training data in adversarial training. It is obvious that using adversarial examples crafted from the private training data to train our models introduces a previously unknown privacy risk, disclosing the participation of the benign examples . (b) There is an unrevealed interplay among DP preservation, adversarial learning, and robustness bounds. (c) Existing algorithms cannot be readily applied to address the trade-off among model utility, privacy loss, and robustness. Therefore, theoretically bounding the robustness of a model (which both protects the privacy and is robust against adversarial examples) is nontrivial. Motivated by this open problem, we propose to develop a novel differentially private adversarial learning (DPAL) mechanism to: 1) preserve DP of the training data, 2) be provably and practically robust to adversarial examples, and 3) retain high model utility. In our mech-anism, privacy-preserving noise is injected into inputs and hidden layers to achieve DP in learning private model parameters (Theorem 1). Then, we incorporate ensemble adversarial learning into our mechanism to improve the decision boundary under DP protections. To do this, we introduce a concept of DP adversarial examples crafted using benign examples in the private training data under DP guarantees (Eq. 9). To address the trade-off between model utility and privacy loss, we propose a new DP adversarial objective function to tighten the model's global sensitivity (Theorem 3); thus, we significantly reduce the amount of noise injected into our function, compared with existing works (; 2017b; a). In addition, ensemble DP adversarial examples with a dynamic perturbation size µ a are introduced into the training process to further improve the robustness of our mechanism under different attack algorithms. An end-to-end privacy analysis shows that, by slitting the private training data into disjoint and fixed batches across epochs, the privacy budget in our DPAL is not accumulated across training steps (Theorem 4). After preserving DP in learning model parameters, we establish a solid connection among privacy preservation, adversarial learning, and provable robustness. Noise injected into different layers is considered as a sequence of randomizing mechanisms, providing different levels of robustness. By leveraging the sequential composition theory in DP , we derive a novel generalized robustness bound, which essentially is a composition of these levels of robustness (Theorem 5 and Proposition 1). To our knowledge, our mechanism establishes the first connection between DP preservation and provable robustness against adversarial examples in adversarial learning. Rigorous experiments conducted on MNIST and CIFAR-10 datasets show that our mechanism notably enhances the robustness of DP deep neural networks, compared with existing mechanisms. In this section, we revisit adversarial learning, DP, and our problem definition. Let D be a database that contains N tuples, each of which contains data x ∈ [−1, 1] d and a ground-truth label y ∈ Z K, with K possible categorical outcomes. Each y is a one-hot vector of K categories y = {y 1, . . ., y K}. A single true class label y x ∈ y given x ∈ D is assigned to only one of the K categories. On input x and parameters θ, a model outputs class scores f: R d → R K that maps d-dimensional inputs x to a vector of scores f (x) = {f 1 (x),..., f K (x)} s.t. ∀k ∈ [1, K]: f k (x) ∈ and K k=1 f k (x) = 1. The class with the highest score value is selected as the predicted label for the data tuple, denoted as y(x) = max k∈K f k (x). A loss function L(f (x), y) presents the penalty for mismatching between the predicted values f (x) and original values y. For the sake of clarity, the notations and terminologies frequently used in this paper are summarized in Table 1 (Appendix A). Let us briefly revisit DP-preserving techniques in deep learning, starting with the definition of DP. Definition 1 (, δ)-DP . A randomized algorithm A fulfills (, δ)-DP, if for any two databases D and D differing at most one tuple, and for all O ⊆ Range(A), we have: A smaller enforces a stronger privacy guarantee. Here, controls the amount by which the distributions induced by D and D may differ, δ is a broken probability. DP also applies to general metrics ρ(D, D) ≤ 1, where ρ can be l p -norms . DP-preserving algorithms in deep learning can be categorized into two lines: 1) introducing noise into gradients of parameters (; ; ; ; ;), 2) injecting noise into objective functions (; 2017b; a), and 3) injecting noise into labels . In Lemmas 2 and 4, we will show that our mechanism achieves better sensitivity bounds compared with existing works (; 2017b; a). Adversarial Learning. For some target model f and inputs (x, y x), the adversary's goal is to find an adversarial example x adv = x + α, where α is the perturbation introduced by the attacker, such that: x adv and x are close, and the model misclassifies x adv, i.e., y(x adv) = y(x). In this paper, we consider well-known l p∈{1,2,∞} -norm bounded attacks (b). Let l p (µ) = {α ∈ R d : α p ≤ µ} be the l p -norm ball of radius µ. One of the goals in adversarial learning is to minimize the risk over adversarial examples: θ * = arg min θ E (x,ytrue)∼D max α p ≤µ L f (x + α, θ), y x, where an attack is used to approximate solutions to the inner maximization problem, and the outer minimization problem corresponds to training the model f with parameters θ over these adversarial examples x adv = x + α. There are two basic adversarial example attacks. The first one is a single-step algorithm, in which only a single gradient computation is required. For instance, FGSM algorithm (b) finds adversarial examples by solving the inner maximization max α p ≤µ L f (x + α, θ), y x. The second one is an iterative algorithm, in which multiple gradients are computed and updated. For instance, in (a), FGSM is applied multiple times with T µ small steps, each of which has a size of µ/T µ. To improve the robustness of models, prior work focused on two directions: 1) Producing correct predictions on adversarial examples, while not compromising the accuracy on legitimate inputs (; ; ; b; a; ; ;); and 2) Detecting adversarial examples (; ; ; Abbasi & Gagné, 2017;). Among existing solutions, adversarial training appears to hold the greatest promise for learning robust models (Tramèr et al., 2017). One of the well-known algorithms was proposed in (b). At every training step, new adversarial examples are generated and injected into batches containing both benign and adversarial examples. The typical adversarial learning in (b) is presented in Alg. 2 (Appendix B). DP and Provable Robustness. Recently, some algorithms (; ; ; ;) have been proposed to derive provable robustness, in which each prediction is guaranteed to be consistent under the perturbation α, if a robustness condition is held. Given a benign example x, we focus on achieving a robustness condition to attacks of l p (µ)-norm, as follows: where k = y(x), indicating that a small perturbation α in the input does not change the predicted label y(x). To achieve the robustness condition in Eq. 2, Lecuyer et al. introduce an algorithm, called PixelDP. By considering an input x (e.g., images) as databases in DP parlance, and individual features (e.g., pixels) as tuples, PixelDP shows that randomizing the scoring function f (x) to enforce DP on a small number of pixels in an image guarantees robustness of predictions against adversarial examples. To randomize f (x), random noise σ r is injected into either input x or an arbitrary hidden layer, ing in the following (r, δ r)-PixelDP condition: Lemma 1 (r, δ r)-PixelDP . Given a randomized scoring function f (x) satisfying (r, δ r)-PixelDP w.r.t. a l p -norm metric, we have: is the expected value of f k (x), r is a predefined budget, δ r is a broken probability. At the prediction time, a certified robustness check is implemented for each prediction. A generalized robustness condition is proposed as follows: whereÊ lb andÊ ub are the lower and upper bounds of the expected valueÊf (x) = 1 n n f (x) n, derived from the Monte Carlo estimation with an η-confidence, given n is the number of invocations of f (x) with independent draws in the noise σ r. Passing the check for a given input guarantees that no perturbation up to l p-norm can change the model's prediction. PixelDP does not preserve DP in learning private parameters θ to protect the training data. That is different from our goal. Our new DPAL mechanism is presented in Alg. 1. Our network (Figure 1) can be represented as: f (x) = g(a(x, θ 1), θ 2 ), where a(x, θ 1) is a feature representation learning model with x as an input, and g will take the output of a(x, θ 1) and return the class scores f (x). At a high level, DPAL has three key components: DP a(x, θ 1), which is to preserve DP in learning the feature representation model a(x, θ 1); DP Adversarial Learning, which focuses on preserving DP in adversarial learning, given DP a(x, θ 1); and Provable Robustness and Verified Inferring, which are to compute robustness bounds given an input at the inference time. In particular, given a deep neural network f with model parameters θ (Lines 2-3), the network is trained over T training steps. In each step, a batch of m perturbed training examples and a batch of m DP adversarial examples derived from D are used to train our network Take a batch Bi ∈ B where i = t%(N/m), Assign Bt ← Bi 6: Ensemble DP Adversarial Examples: Take a batch Bi+1 ∈ B, Assign B adv t ← ∅ 9: for l ∈ A do 10: Take the next batch Ba ⊂ Bi+1 with the size m/|A| 11: ∀xj ∈ Ba: Craft x adv j by using attack algorithm (θ2) with the noise Output: (1 + 1/γx + 1/γ + 2)-DP parameters θ = {θ1, θ2}, robust model with an r budget 13: Verified Inferring: (an input x, attack size µa) 14: Compute robustness size (κ + ϕ)max in Eq. 15 of x 15: if (κ + ϕ)max ≥ µa then 16: Return isRobust(x) = T rue, label k, (κ + ϕ)max 17: else 18: Return isRobust(x) = F alse, label k, (κ + ϕ)max 3.1 DP FEATURE REPRESENTATION LEARNING Our idea is to use auto-encoder to simultaneously learn DP parameters θ 1 and ensure that the output of a(x, θ 1) is DP. The reasons we choose an auto-encoder are: It is easier to train, given its small size; and It can be reused for different predictive models. A typical data reconstruction function (cross-entropy), given a batch B t at the training step t of the input x i, is as follows: where the transformation of x i is h i = θ T 1 x i, the hidden layer h 1 of a(x, θ 1) given the batch B t is denoted as h 1Bt = {θ T 1 x i} xi∈Bt, and x i = θ 1 h i is the reconstruction of x i. To preserve 1 -DP in learning θ 1 where 1 is a privacy budget, we first derive the 1st-order polynomial approximation of R Bt (θ 1) by applying Taylor Expansion , denoted as R Bt (θ 1). Then, Functional Mechanism is employed to inject noise into coefficients of the approximated function where, parameters θ 1j derived from the function optimization need to be 1 -DP. To achieve that, Laplace noise ) is injected into coefficients 1 2 − x ij h i, where ∆ R is the sensitivity of R Bt (θ 1), as follows: To ensure that the computation of x i does not access the original data, we further inject Laplace noise ) into x i. This can be done as a preprocessing step for all the benign examples in D to construct a set of disjoint batches B of perturbed benign examples (Lines 2 and 5). The perturbed function now becomes: where, and x i = θ 1 h i. Let us denote β as the number of neurons in h 1, and h i is bounded in [−1, 1], the global sensitivity ∆ R is as follows: Lemma 2 The global sensitivity of R over any two neighboring batches, B t and B t, is as follows: All the proofs are in our Appendix. By setting ∆ R = d(β + 2), we show that the output of a(·), which is the perturbed affine transformation h 1Bt = {θ and θ 1 1,1 is the maximum 1-norm of θ 1 's columns (Operator norm, 2018). This is important to tighten the privacy budget consumption in computing the remaining hidden layers g(a(x, θ 1), θ 2 ). In fact, without using additional information from the original data, the computation of g(a(x, θ 1), θ 2 ) is also (1 /γ)-DP (the post-processing property of DP). Similarly, we observe that the perturbation of a batch Note that we do not use the post-processing property of DP to estimate the DP guarantee of h 1Bt based upon the DP guarantee of B t, since 1 /γ < 1 /γ x in practice. As a , the (1 /γ)-DP h 1Bt provides a more rigorous DP protection to the computation of g(·) and to the output layer. Lemma 3 The computation of the affine transformation h 1Bt is (1 /γ)-DP and the computation of the batch B t as the input layer is (1 /γ x)-DP. The following Theorem shows that optimizing R Bt (θ 1) is (1 /γ x + 1)-DP in learning θ 1 given an (1 /γ x)-DP B t batch. The optimization of R Bt (θ 1) preserves (1 /γ x + 1)-DP in learning θ 1. To integrate adversarial learning, we first draft DP adversarial examples x adv j using perturbed benign examples x j, with an ensemble of attack algorithms A and a random perturbation budget µ t ∈, at each step t (Lines 6-11). This will significantly enhances the robustness of our models under different types of adversarial examples with an unknown adversarial attack size µ. with y(x j) is the class prediction of f (x j) to avoid label leaking of the benign examples x j during the adversarial example crafting. Given a set of DP adversarial examples B adv t, training the auto-encoder with B adv t preserves (1 /γ x + 1)-DP. The proof of Theorem 2 is in Appendix H, Result 4. It can be extended to iterative attacks as where y(x . Second, we propose a novel DP adversarial objective function L Bt (θ 2), in which the loss function L for benign examples is combined with an additional loss function Υ for DP adversarial examples, to optimize the parameters θ 2. The objective function L Bt (θ 2) is defined as follows: where ξ is a hyper-parameter. For the sake of clarity, in Eq. 10, we denote y i and y j as the true class labels y xi and y xj of examples x i and x j. Note that x adv j and x j share the same label y xj. Now we are ready to preserve DP in objective functions L f (x i, θ 2), y i and Υ f (x adv j, θ 2), y j in order to achieve DP in learning θ 2. Since the objective functions use the true class labels y i and y j, we need to protect the labels at the output layer. Let us first present our approach to preserve DP in the objective function L for benign examples. Given h πi computed from the x i through the network with W π is the parameter at the last hidden layer h π. Cross-entropy function is approximated as Based on the post-processing property of DP ,. As a , the optimization of the function L 1Bt θ 2 does not disclose any information from the training data, and 1/γ, given neighboring batches B t and B t. Thus, we only need to preserve 2 -DP in the function L 2Bt (θ 2), which accesses the ground-truth label y ik. Given coefficients h πi y ik, the sensitivity ∆ L2 of L 2Bt (θ 2) is computed as: Lemma 4 Let B t and B t be neighboring batches of benign examples, we have the following inequality: ∆ L2 ≤ 2|h π |, where |h π | is the number of hidden neurons in h π. The sensitivity of our objective function is notably smaller than the state-of-the-art bound (a), which is crucial to improve our model utility. The perturbed functions are as follows: We apply the same technique to preserve As the perturbed functions L and Υ are always optimized given two disjoint batches B t and B adv t, the privacy budget used to preserve DP in the adversarial objective function L Bt (θ 2) is (1 /γ + 2), following the parallel composition property of DP . The total budget to learn private parameters We have shown that our mechanism achieves DP at the batch level B t ∪B adv t given a specific training step t. By constructing disjoint and fixed batches from the training data D, we leverage both parallel composition and post-processing properties of DP to extend the to (1 + 1 /γ x + 1 /γ + 2)-DP in learning θ = {θ 1, θ 2} on D across T training steps. There are three key properties in our approach: It only reads perturbed inputs B t and perturbed coefficients h 1, which are DP across T training steps; Given N/m disjoint batches in each epoch, for any example x, x is included in one and only one batch, denoted B x ∈ B. As a , the DP guarantee to x in D is equivalent to the DP guarantee to x in B x; since the optimization using any other batches does not affect the DP guarantee of x; and All the batches are fixed across T training steps to prevent additional privacy leakage, caused by generating new and overlapping batches (which are considered overlapping datasets in the parlance of DP) in the typical training approach. Theorem 4 Algorithm 1 achieves (1 + 1 /γ x + 1 /γ + 2)-DP parameters θ = {θ 1, θ 2} on the private training data D across T training steps. Now, we establish the correlation between our mechanism and provable robustness. In the inference time, to derive the provable robustness condition against adversarial examples x+α, i.e., ∀α ∈ l p, PixelDP mechanism randomizes the scoring function f (x) by injecting robustness noise σ r into either input x or a hidden layer, i.e., x = x + Lap(, where ∆ x r and ∆ h r are the sensitivities of x and h, measuring how much x and h can be changed given the perturbation α ∈ l p in the input x. Monte Carlo estimation of the expected valuesÊf (x),Ê lb f k (x), and E ub f k (x) are used to derive the robustness condition in Eq. 4. On the other hand, in our mechanism, the privacy noise σ p includes Laplace noise injected into both input x, i.e., This helps us to avoid injecting the noise directly into the coefficients h πi y ik. The correlation between our DP preservation and provable robustness lies in the correlation between the privacy noise σ p and the robustness noise σ r. We can derive a robustness bound by projecting the privacy noise σ p on the scale of the robustness noise σ r. Given the input x, let κ =, in our mechanism we have that: By applying a group privacy size κ , the scoring function f (x) satisfies r -PixelDP given α ∈ l p (κ), or equivalently is κ r -PixelDP given α ∈ l p, δ r = 0. By applying Lemma 1, we have ∀k, ∀α ∈ l p (κ): With that, we can achieve a robustness condition against l p (κ)-norm attacks, as follows: with the probability ≥ η x -confidence, derived from the Monte Carlo estimation ofÊf (x). Our mechanism also perturbs h (Eq. 7). Given ϕ = ). Therefore, the scoring function f (x) also satisfies r -PixelDP given the perturbation α ∈ l p (ϕ). In addition to the robustness to the l p (κ)-norm attacks, we achieve an additional robustness bound in Eq. 12 against l p (ϕ)-norm attacks. Similar to PixelDP, these robustness conditions can be achieved as randomization processes in the inference time. They can be considered as two independent and provable defensive mechanisms applied against two l p -norm attacks, i.e., l p (κ) and l p (ϕ). One challenging question here is: "What is the general robustness bound, given κ and ϕ?" Intuitively, our model is robust to attacks with α ∈ l p (κ + ϕ). We leverage the theory of sequential composition in DP to theoretically answer this question. Given S independent mechanisms M 1,..., M S, whose privacy guarantees are 1,..., S -DP with α ∈ l p. Each mechanism M s, which takes the input x and outputs the value of f (x) with the Laplace noise only injected to randomize the layer s (i.e., no randomization at any other layers), denoted as f s (x), is defined as: We aim to derive a generalized robustness of any composition scoring function f (M 1, . . ., M s |x) bounded in, defined as follows: Our setting follows the sequential composition in DP . Thus, we can prove that the expected value Ef (M 1, . . ., M S |x) is insensitive to small perturbations α ∈ l p in Lemma 5, and we derive our composition of robustness in Theorem 5, as follows: Lemma 5 Given S independent mechanisms M 1,..., M S, which are 1,..., S -DP w.r.t a l p -norm metric, then the expected output value of any sequential function f of them, i.e., f (M 1, . . ., M S |x) ∈, meets the following property: Theorem 5 (Composition of Robustness) Given S independent mechanisms M 1,..., M S. Given any sequential function f (M 1, . . ., M S |x), and letÊ lb andÊ ub are lower and upper bounds with an η-confidence, for the Monte Carlo estimation ofÊf then the predicted label k = arg max kÊ f k (M 1, . . ., M S |x), is robust to adversarial examples x + α, ∀α ∈ l p, with probability ≥ η, by satisfying:, which is the targeted robustness condition in Eq. 2. It is worth noting that there is no η s -confidence for each mechanism s, since we do not estimate the expected valueÊf s (x) independently. To apply the composition of robustness in our mechanism, the noise injections into the input x and its affine transformation h can be considered as two mechanisms M x and M h, sequentially applied as with independent draws in the noise χ 2, the noise χ 1 injected into x is fixed; and vice-versa. By applying group privacy with sizes κ and ϕ, the scoring functions f x (x) and f h (x), given M x and M h, are κ r -DP and ϕ r -DP given α ∈ l p. With Theorem 5, we have a generalized bound as follows: e., Eq. 14), then the predicted label k of our function f (M h, M x |x) is robust to perturbations α ∈ l p (κ + ϕ) with the probability ≥ η, by satisfying Our model is trained similarly to training typical deep neural networks. Parameters θ 1 and θ 2 are independently updated by applying gradient descent (Line 12). Regarding the inference time, we implement a verified inference procedure as a post-processing step (Lines 13-18). Our verified inference returns a robustness size guarantee for each example x, which is the maximal value of κ + ϕ, for which the robustness condition in Proposition 1 holds. Maximizing κ + ϕ is equivalent to maximizing the robustness epsilon r, which is the only parameter controlling the size of κ + ϕ; since, all the other hyper-parameters, i.e., ∆ R, m, 1, 2, θ 1, θ 2, ∆ x r, and ∆ h r are fixed given a well-trained model f (x): e., Eq. 14) The prediction on an example x is robust to attacks up to (κ + ϕ) max. The failure probability 1-η can be made arbitrarily small by increasing the number of invocations of f (x), with independent draws in the noise. Similar to , Hoeffding's inequality is applied to bound the approximation error inÊf k (x) and to search for the robustness bound (κ + ϕ) max. We use the following sensitivity bounds ∆ h r = β θ 1 ∞ where θ 1 ∞ is the maximum 1-norm of θ 1's rows, and ∆ x r = µd for l ∞ attacks. We also propose a new way to draw independent noise following the distribution of χ 1 + /ψ) for the transformation h, where χ 1 and χ 2 are the fixed noise used to train the network, and ψ is a parameter to control the distribution shifts between training and inferring. This new Monte Carlo Estimation of Ef (x) works better without affecting the DP bounds and the robustness (Appendix L). We have conducted an extensive experiment on the MNIST and CIFAR-10 datasets. We consider the class of l ∞ -bounded adversaries to see whether our mechanism could retain high model utility, while providing strong DP guarantees and protections against adversarial examples. Baseline Approaches. Our DPAL mechanism is evaluated in comparison with state-of-the-art mechanisms in: DP-preserving algorithms in deep learning, i.e., DP-SGD , AdLM (a); in Provable robustness, i.e., PixelDP ; and in DP-preserving algorithms with provable robustness, i.e., SecureSGD given heterogeneous noise , and SecureSGD-AGM given the Analytic Gaussian Mechanism (AGM) . To preserve DP, DP-SGD injects random noise into gradients of parameters, while AdLM is a Functional Mechanism-based approach. PixelDP is one of the state-ofthe-art mechanisms providing provable robustness using DP bounds. SecureSGD is a combination of PixelDP and DP-SGD with an advanced heterogeneous noise distribution; i.e., "more noise" is injected into "more vulnerable" latent features, to improve the robustness. The baseline models share the same design in our experiment. Four white-box attacks were used, including FGSM, I-FGSM, Momentum Iterative Method (MIM) , and MadryEtAl . is equivalent to an attack size 2µ a = 0.6 in our setting. The reason for using x ∈ [−1, 1] d is to achieve better model utility, while retaining the same global sensitivities to preserve DP, compared with x ∈ d. Our model configurations are in Appendix M and our approximation error bound analysis is presented in Appendix N. As in , we apply two accuracy metrics: |test| where |test| is the number of test cases, isCorrect(·) returns 1 if the model makes a correct prediction (else, returns 0), and isRobust(·) returns 1 if the robustness size is larger than a given attack size µ a (else, returns 0). Our task of validation focuses on shedding light into the interplay among model utility, privacy loss, and robustness bounds, by learning 1) the impact of the privacy budget t = (1 + 1 /γ x + 1 /γ + 2), and 2) the impact of attack sizes µ a. All statistical tests are 2-tail t-tests. All experimental Figures are in Appendix O. Results on the MNIST Dataset. Figure 2 illustrates the conventional accuracy of each model as a function of the privacy budget t on the MNIST dataset under l ∞ (µ a)-norm attacks, with µ a = 0.2 (a pretty strong attack). It is clear that our DPAL outperforms AdLM, DP-SGD, SecureSGD, and SecureSGD-AGM, in all cases, with p < 1.32e − 4. On average, we register a 22.36% improvement over SecureSGD (p < 1.32e − 4), a 46.84% improvement over SecureSGD-AGM (p < 1.83e − 6), a 56.21% improvement over AdLM (p < 2.05e − 10), and a 77.26% improvement over DP-SGD (p < 5.20e − 14), given our DPAL mechanism. AdLM and DP-SGD achieve the worst conventional accuracies. There is no guarantee provided in AdLM and DP-SGD. Thus, the accuracy of the AdLM and DPSGD algorithms seem to show no effect against adversarial examples, when the privacy budget is varied. This is in contrast to our DPAL model, the SecureSGD model, and the SecureSGD-AGM model, whose accuracies are proportional to the privacy budget. When the privacy budget t = 0.2 (a tight DP protection), there are significant drops, in terms of conventional accuracy, given the baseline approaches. By contrast, our DPAL mechanism only shows a small degradation in the conventional accuracy (6.89%, from 89.59% to 82.7%), compared with a 37% drop in SecureSGD (from 78.64% to 41.64%), and a 32.89% drop in SecureSGD-AGM (from 44.1% to 11.2%) on average, when the privacy budget t goes from 2.0 to 0.2. At t = 0.2, our DPAL mechanism achieves 82.7%, compared with 11.2% and 41.64% correspondingly for SecureSGD-AGM and SecureSGD. This is an important , showing the ability to offer tight DP protections under adversarial example attacks in our model, compared with existing algorithms. • Figure 4 presents the conventional accuracy of each model as a function of the attack size µ a on the MNIST dataset, under a strong DP guarantee, t = 0.2. It is clear that our DPAL mechanism outperforms the baseline approaches in all cases. On average, our DPAL model improves 44.91% over SecureSGD (p < 7.43e − 31), a 61.13% over SecureSGD-AGM (p < 2.56e − 22), a 52.21% over AdLM (p < 2.81e − 23), and a 62.20% over DP-SGD (p < 2.57e − 22). More importantly, our DPAL model is resistant to different adversarial example algorithms with different attack sizes. When µ a ≥ 0.2, AdLM, DP-SGD, SecureSGD, and SecureSGD-AGM become defenseless. We further register significantly drops in terms of accuracy, when µ a is increased from 0.05 (a weak attack) to 0.6 (a strong attack), i.e., 19.87% on average given our DPAL, across all attacks, compared with 27.76% (AdLM), 29.79% (DP-SGD), 34.14% (SecureSGD-AGM), and 17.07% (SecureSGD). • Figure 6 demonstrates the certified accuracy as a function of µ a. The privacy budget is set to 1.0, offering a reasonable privacy protection. In PixelDP, the construction attack bound r is set to 0.1, which is a pretty reasonable defense. With (small perturbation) µ a ≤ 0.2, PixelDP achieves better certified accuracies under all attacks; since PixelDP does not preserve DP to protect the training data, compared with other models. Meanwhile, our DPAL model outperforms all the other models when µ a ≥ 0.3, indicating a stronger defense to more aggressive attacks. More importantly, our DPAL has a consistent certified accuracy to different attacks given different attack sizes, compared with baseline approaches. In fact, when µ a is increased from 0.05 to 0.6, our DPAL shows a small drop (11.88% on average, from 84.29%(µ a = 0.05) to 72.41%(µ a = 0.6)), compared with a huge drop of the PixelDP, i.e., from 94.19%(µ a = 0.05) to 9.08%(µ a = 0.6) on average under I-FGSM, MIM, and MadryEtAl attacks, and to 77.47%(µ a = 0.6) under FGSM attack. Similarly, we also register significant drops in terms of certified accuracy for SecureSGD (78.74%, from 86.74% to 7.99%) and SecureSGD-AGM (81.97%, from 87.23% to 5.26%) on average. This is promising. Our key observations are as follows. Incorporating ensemble adversarial learning into DP preservation, with tightened sensitivity bounds and a random perturbation size µ t ∈ at each training step, does enhance the consistency, robustness, and accuracy of our model against different attack algorithms with different levels of perturbations. Our DPAL model outperforms baseline algorithms, including both DP-preserving and non-private approaches, in terms of conventional accuracy and certified accuracy in most of the cases. It is clear that existing DP-preserving approaches have not been designed to withstand against adversarial examples. Results on the CIFAR-10 Dataset further strengthen our observations. In Figure 3, our DPAL clearly outperforms baseline models in all cases (p < 6.17e−9), especially when the privacy budget is small (t < 4), yielding strong privacy protections. On average conventional accuracy, our DPAL mechanism has an improvement of 10.42% over SecureSGD (p < 2.59e − 7), an improvement of 14.08% over SecureSGD-AGM (p < 5.03e − 9), an improvement of 29.22% over AdLM (p < 5.28e − 26), and a 14.62% improvement over DP-SGD (p < 4.31e − 9). When the privacy budget is increased from 2 to 10, the conventional accuracy of our DPAL model increases from 42.02% to 46.76%, showing a 4.74% improvement on average. However, the conventional accuracy of our model under adversarial example attacks is still low, i.e., 44.22% on average given the privacy budget at 2.0. This opens a long-term research avenue to achieve better robustness under strong privacy guarantees in adversarial learning. • The accuracy of our model is consistent given different attacks with different adversarial perturbations µ a under a rigorous DP protection (t = 2.0), compared with baseline approaches (Figure 5). In fact, when the attack size µ a increases from 0.05 to 0.5, the conventional accuracies of the baseline approaches are remarkably reduced, i.e., a drop of 25.26% on average given the most effective baseline approach, SecureSGD. Meanwhile, there is a much smaller degradation (4.79% on average) in terms of the conventional accuracy observed in our DPAL model. Our model also achieves better accuracies compared with baseline approaches in all cases (p < 8.2e − 10). Figure 7 further shows that our DPAL model is more accurate than baseline approaches (i.e., r is set to 0.1 in PixelDP) in terms of certified accuracy in all cases, with a tight privacy budget set to 2.0 (p < 2.04e − 18). We register an improvement of 21.01% in our DPAL model given the certified accuracy over SecureSGD model, which is the most effective baseline approach (p < 2.04e − 18). In this paper, we established a connection among DP preservation to protect the training data, adversarial learning, and provable robustness. A sequential composition robustness theory was introduced to generalize robustness given any sequential and bounded function of independent defensive mechanisms. An original DP-preserving mechanism was designed to address the trade-off among model utility, privacy loss, and robustness by tightening the global sensitivity bounds. A new Monte Carlo Estimation was proposed to improve and stabilize the estimation of the robustness bounds; thus improving the certified accuracy under adversarial example attacks. However, there are several limitations. First, the accuracy of our model under adversarial example attacks is still very low. Second, the mechanism scalability is dependent on the model structures. Third, further study is needed to address the threats from adversarial examples crafted by unseen attack algorithms. Fourth, in this study, our goal is to illustrate the difficulties in providing DP protections to the training data in adversarial learning with robustness bounds. The problem is more challenging when working with complex and large networks, such as ResNet , VGG16 , LSTM , and GAN (a). Fifth, there can be alternative approaches to draft and to use DP adversarial examples. Addressing these limitations needs significant efforts from both research and practice communities. A NOTATIONS AND TERMINOLOGIES Function/model f that maps inputs x to a vector of scores f (x) = {f1(x),..., fK (x)} yx ∈ y A single true class label of example x y(x) = max k∈K f k (x) Predicted label for the example x given the function f x adv = x + α Adversarial example where α is the perturbation lp(µ) = {α ∈ R d : α p ≤ µ} The lp-norm ball of attack radius µ (r, δr) Robustness budget r and broken probability δr The expected value of f k (x) E lb andÊ ub Lower and upper bounds of the expected valueÊf (x) = Feature representation learning model with x and parameters θ1 Bt A batch of benign examples xi Data reconstruction function given Bt in a(x, θ1) The values of all hidden neurons in the hidden layer h1 of a(x, θ1) given the batch Bt RB t (θ1) and R B t (θ1) Approximated and perturbed functions of RB t (θ1) xi and xi Perturbed and reconstructed inputs xi Sensitivity of the approximated function RB t (θ1) h1B Sensitivities of x and h, given the perturbation α ∈ lp Privacy budget to protect the training data D (κ + ϕ)max Robustness size guarantee given an input x at the inference time B PSEUDO-CODE OF ADVERSARIAL TRAINING (KURAKIN ET AL., 2016B) Given a loss function: where m 1 and m 2 correspondingly are the numbers of examples in B t and B adv t at each training step. Proof 1 Assume that B t and B t differ in the last tuple, x m (x m). Then, Proof 2 Regarding the computation of h 1Bt = {θ The sensitivity of a function h is defined as the maximum change in output, that can be generated by a change in the input . Therefore, the global sensitivity of h 1 can be computed as follows: following matrix norms (Operator norm, 2018): θ T 1 1,1 is the maximum 1-norm of θ 1's columns. By injecting Laplace noise Lap(, and χ 2 drawn as a Laplace noise [Lap( β, in our mechanism, the perturbed affine transformation h 1Bt is presented as: This in an ( 1 /γ)-DP affine transformation h 1Bt = {θ Similarly, the perturbed inputs where ∆ x is the sensitivity measuring the maximum change in the input layer that can be generated by a change in the batch B t and γ x = ∆ R m∆x . Following , ∆ x can be computed as follows: Consequently, Lemma 3 does hold. Proof 3 Given χ 1 drawn as a Laplace noise [Lap( d and χ 2 drawn as a Laplace noise β, the perturbation of the coefficient φ ∈ Φ = { 1 2 h i, x i}, denoted as φ, can be rewritten as follows:, we have that: Consequently, the computation of R Bt (θ 1) preserves 1 -DP in Alg. 1. In addition, the parameter optimization of R Bt (θ 1) only uses the perturbed data B t, which is (1 /γ x)-DP (Lemma 3), in the computations of h i, h i, x i, parameter gradients, and gradient descents at each step. These operations do not access the original dataset B t; therefore, they do not incur any additional information from the original data (the post-processing property in). As a , the total privacy budget to learn the perturbed optimal parameters θ 1 in Alg. 1 is (1 /γ x + 1)-DP. Proof 4 Assume that B t and B t differ in the last tuple, and x m (x m) be the last tuple in B t (B t), we have that Since y mk and y mk are one-hot encoding, we have that Proof 5 Let B t and B t be neighboring batches of benign examples, and χ 3 drawn as Laplace noise [Lap( |hπ|, the perturbations of the coefficients h πi y ik can be rewritten as: Since all the coefficients are perturbed, and given ∆ L2 = 2|h π |, we have that The computation of L 2Bt θ 2 preserves ( 1 /γ + 2)-differential privacy. The optimization of L 2Bt θ 2 does not access additional information from the original input x i ∈ B t. Consequently, the optimal perturbed parameters θ 2 derived from L 2Bt θ 2 are (1 /γ + 2)-DP. Proof 6 First, we optimize for a single draw of noise during training (Line 3) and all the batches of perturbed benign examples are disjoint and fixed across epochs. As a , the computation of x i is equivalent to a data preprocessing step with DP, which does not incur any additional privacy budget consumption over T training steps (the post-processing property of DP) (Result 1). That is different from repeatedly applying a DP mechanism on either the same or overlapping datasets causing the accumulation of the privacy budget. Now, we show that our algorithm achieves DP at the dataset level D. Let us consider the computation of the first hidden layer, given any two neighboring datasets D and D differing at most one tuple x e ∈ D and x e ∈ D., we have that By having disjoint and fixed batches, we have that: From Eqs. 19, 20, and Lemma 3, we have that Eqs. 20 and 21 As a , the computation of h 1D is (1 /γ)-DP given the data D, since the Eq. 22 does hold for any tuple x e ∈ D. That is consistent with the parallel composition property of DP, in which batches can be considered disjoint datasets given h 1B as a DP mechanism . This does hold across epochs, since batches B are disjoint and fixed among epochs. At each training step t ∈ [1, T], the computation of h 1Bt does not access the original data. It only reads the perturbed batch of inputs B t, which is (1 /γ x)-DP (Lemma 3). Following the post-processing property in DP , the computation of h 1Bt does not incur any additional information from the original data across T training steps. Similarly, we show that the optimization of the function R Bt (θ 1) is (1 /γ x + 1)-DP across T training steps. As in Theorem 1 and Proof 3, we have that, where B ∈ B. Given any two perturbed neighboring datasets D and D differing at most one tuple x e ∈ D and x e ∈ D: From Eqs. 20, 23, and Theorem 1, we have that Eqs. 23 and 24 As a , the optimization of R D (θ 1) is (1 /γ x + 1)-DP given the data D (which is 1 /γ x -DP (Lemma 3)), since the Eq. 25 does hold for any tuple x e ∈ D. This is consistent with the parallel composition property in DP , in which batches can be considered disjoint datasets and the optimization of the function on one batch does not affect the privacy guarantee in any other batch. In addition, ∀t ∈ [1, T], the optimization of R Bt (θ 1) does not use any additional information from the original data D. Consequently, the privacy budget is (1 /γ x + 1) across T training steps, following the post-processing property in DP Similarly, we can also prove that optimizing the data reconstruction function R B adv t (θ 1) given the DP adversarial examples crafted in Eqs. 8 and 9, i.e., x adv j, is also (1 /γ x + 1)-DP given t ∈ [1, T] on the training data D. First, DP adversarial examples x adv j are crafted from perturbed benign examples x j. As a , the computation of the batch B adv t of DP adversarial examples is 1) (1 /γ x)-DP (the post-processing property of DP ), and 2) does not access the original data ∀t ∈ [1, T]. In addition, the computation of h 1B adv t and the optimization of R B adv t (θ 1) correspondingly are 1 /γ-DP and 1 -DP. In fact, the data reconstruction function R B adv t is presented as follows: where h, and x adv j = θ 1 h adv j. The right summation component in Eq. 26 does not disclose any additional information, since the sign(·) function is computed from perturbed benign examples (the post-processing property in DP ). Meanwhile, the left summation component has the same form with R Bt (θ 1) in Eq. 7. Therefore, we can employ the Proof 3 in Theorem 1, by replacing the coefficients Φ = {In addition to the Result 4, by applying the same analysis in Result 3, we can further show that the optimization of R D adv (θ 1) is (1 /γ x + 1)-DP given the DP adversarial examples D adv crafted using the data D across T training steps, since batches used to created DP adversarial examples are disjoint and fixed across epochs. It is also straightforward to conduct the same analysis in Result 2, in order to prove that the computation of the first affine transformation h 1B given the batch of DP adversarial examples B Regarding the output layer, the Algorithm 1 preserves (1 /γ + 2)-DP in optimizing the adversarial objective function L Bt∪B adv t (θ 2) (Theorem 3). We apply the same technique to preserve (1 /γ + 2)-DP across T training steps given disjoint and fixed batches derived from the private training data D. In addition, as our objective functions R and L are always optimized given two disjoint batches B t and B adv t, the privacy budget used to preserve DP in these functions is (1 + 1 /γ + 2), following the parallel composition property in DP With the Results 1-6, all the computations and optimizations in the Algorithm 1 are DP following the post-processing property in DP , by working on perturbed inputs and perturbed coefficients. The crafting and utilizing processes of DP adversarial examples based on the perturbed benign examples do not disclose any additional information. The optimization of our DP adversarial objective function at the output layer is DP to protect the ground-truth labels. More importantly, the DP guarantee in learning given the whole dataset level D is equivalent to the DP guarantee in learning on disjoint and fixed batches across epochs. Consequently, Algorithm 1 preserves (1 + 1 /γ x + 1 /γ + 2)-DP in learning private parameters θ = {θ 1, θ 2} given the training data D across T training steps. Note that the 1 /γ x is counted for the perturbation on the benign examples. Theorem 4 does hold. Proof 7 Thanks to the sequential composition theory in DP , As a , we have The sequential composition of the expected output is as: Lemma 5 does hold. Proof 8 ∀α ∈ l p, from Lemma 5, with probability ≥ η, we have that In addition, we also have Using the hypothesis (Eq. 14) and the first inequality (Eq. 27), we have that Now, we apply the third inequality (Eq. 28), we have that The Theorem 5 does hold. Proof 9 ∀α ∈ l p, by applying Theorem 5, we havê Furthermore, by applying group privacy, we have that By applying Proof 8, it is straight to have with probability ≥ η. Proposition 1 does hold. Recall that the Monte Carlo estimation is applied to estimate the expected valueÊf (x) = 1 n n f (x) n, where n is the number of invocations of f (x) with independent draws in the noise, i.e., ) in our case. When 1 is small (indicating a strong privacy protection), it causes a notably large distribution shift between training and inference, given independent draws of the Laplace noise. In fact, let us denote a single draw in the noise as ) used to train the function f (x), the model converges to the point that the noise χ 1 and 2χ 2 need to be correspondingly added into x and h in order to make correct predictions. χ 1 can be approximated as Lap(χ 1,), where → 0. It is clear that independent draws of the noise Lap(χ 1,). These distribution shifts can also be large, when noise is large. We have experienced that these distribution shifts in having independent draws of noise to estimatê Ef (x) can notably degrade the inference accuracy of the scoring function, when privacy budget 1 is small ing in a large amount of noise injected to provide strong privacy guarantees. To address this, one solution is to increase the number of invocations of f (x), i.e., n, to a huge number per prediction. However, this is impractical in real-world scenarios. We propose a novel way to draw independent noise following the distribution of χ 1 + /ψ) for the affine transformation h, where ψ is a hyper-parameter to control the distribution shifts. This approach works well and does not affect the DP bounds and the provable robustness condition, since: Our mechanism achieves both DP and provable robustness in the training process; and It is clear thatÊf is the n-th draw of the noise. When n → ∞,Êf (x) will converge to 1 n n g a(x + χ 1, θ 1) + 2χ 2, θ 2, which aligns well with the convergence point of the scoring function f (x). Injecting χ 1 and 2χ 2 to x and h during the estimation ofÊf (x) yields better performance, without affecting the DP and the robustness bounds. The MNIST database consists of handwritten digits . Each example is a 28 × 28 size gray-level image. The CIFAR-10 dataset consists of color images belonging to 10 classes, i.e., airplanes, dogs, etc. The dataset is split into 50,000 training samples and 10,000 test samples . The experiments were conducted on a single GPU, i.e., NVIDIA GTX TITAN X, 12 GB with 3,072 CUDA cores. All the models share the same structure, consisting of 2 and 3 convolutional layers, respectively for MNIST and CIFAR-10 datasets. Both fully-connected and convolution layers can be applied in the representation learning model a(x, θ 1). Given convolution layer, the computation of each feature map needs to be DP; since each of them independently reads a local region of input neurons. Therefore, the sensitivity ∆ R can be considered the maximal sensitivity given any single feature map in the first affine transformation layer. In addition, each hidden neuron can only be used to reconstruct a unit patch of input units. That in d (Lemma 2) being the size of the unit patch connected to each hidden neuron, e.g., d = 9 given a 3 × 3 unit patch, and β is the number of hidden neurons in a feature map. MNIST: We used two convolutional layers (32 and 64 features). Each hidden neuron connects with a 5x5 unit patch. A fully-connected layer has 256 units. The batch size m was set to 2,499, ξ = 1, ψ = 2. I-FGSM, MIM, and MadryEtAl were used to draft l ∞ (µ) adversarial examples in training, with T µ = 10. Learning rate t was set to 1e − 4. Given a predefined total privacy budget t, 2 is set to be 0.1, and 1 is computed as: 1 = t− 2 (1+1/γ+1/γx). This will guarantee that (1 + 1 /γ x + 1 /γ + 2) = t. ∆ R = (14 2 + 2) × 25 and ∆ L2 = 2 × 256. We used three convolutional layers (128, 128, and 256 features). Each hidden neuron connects with a 4x4 unit patch in the first layer, and a 5x5 unit patch in other layers. One fullyconnected layer has 256 neurons. The batch size m was set to 1,851, ξ = 1.5, ψ = 10, and T µ = 3. The ensemble of attacks A includes I-FGSM, MIM, and MadryEtAl. We use data augmentation, including random crop, random flip, and random contrast. Learning rate t was set to 5e − 2. In the CIFAR-10 dataset, 2 is set to (1 + r/3.0) and 1 = (1 + 2r/3.0)/(1 + 1/γ + 1/γ x), where r ≥ 0 is a ratio to control the total privacy budget t in our experiment. For instance, given r = 0, we have Computational Efficiency and Scalability. In terms of computation efficiency, our mechanism does not consume any extra computational resources to train the model, compared with existing DP-preserving algorithms in deep learning (; 2017b; a). The model invocations to approximate the robustness bounds can further be efficiently performed in a parallel process. Regarding the scalability, with remarkably tightened global sensitivities, the impact of the size of deep neural networks in terms of the number of hidden layers and hidden neurons is significantly remedied, since 1) ∆ R and ∆ L2 are small, 2) we do not need to inject any noise into the computation of the network g(·), and 3) we do not redraw the noise in each training step t. In addition, our mechanism is not restricted to the type of activation functions. That is similar to . As a , our mechanism has a great potential to be applied in larger deep neural networks using larger datasets. Extensively investigating this property requires further study from both research and practice communities. To compute how much error our polynomial approximation approaches (i.e., truncated Taylor expansions), R Bt (θ 1) (Eq. 6) and L Bt θ 2, incur, we directly apply Lemma 4 in , Lemma 3 in , and the well-known error bound in . Note that R Bt (θ 1) is the 1st-order Taylor series and L Bt θ 2 is the 2nd-order Taylor series. Let us closely follow (; ;) to adapt their into our scenario, as follows: Given the truncated function R Bt (θ 1) = xi∈Bt θ 1j h i r, the average error of the approximation is bounded as where θ 1 = arg min θ1 R Bt (θ 1), θ 1 = arg min θ1 R Bt (θ 1), L Bt (θ 2) is the original Taylor polynomial function of xi∈Bt L f (x i, θ 2), y i, θ 2 = arg min θ2 L Bt (θ 2), θ 2 = arg min θ2 L Bt (θ 2). Proof 10 Let U = max θ1 R Bt (θ 1) − R Bt (θ 1) and S = min θ1 R Bt (θ 1) − R Bt (θ 1). We have that U ≥ R Bt (θ 1) − R Bt (θ 1) and ∀θ * 1: S ≤ R Bt (θ * 1) − R Bt (θ * 1). Therefore, we have In addition, R Bt (θ 1) − R Bt (θ * 1) ≤ 0, it is straightforward to have: If U ≥ 0 and S ≤ 0 then we have: Eq. 34 holds for every θ * 1, including θ 1. Eq. 34 shows that the error incurred by truncating the Taylor series approximate function depends on the maximum and minimum values of R Bt (θ 1) − R Bt (θ 1). This is consistent with . To quantify the magnitude of the error, we rewrite R Bt (θ 1) − R Bt (θ 1) as: where g 1j (x i, θ 1j) = θ 1j h i and g 2j (x i, θ 1j) = θ 1j h i. By looking into the remainder of Taylor expansion for each j (i.e., following ), with z j ∈ [z lj − 1, z lj + 1], 1 |Bt| R Bt (θ 1j) − R Bt (θ 1j) must be in the interval. This can be applied to the case of our autoencoder, as follows: For the functions F 1j (z j) = x ij log(1 + e −zj) and F 2j (z j) = (1 − x ij) log(1 + e zj), we have F Consequently, Eq. 29 does hold. Similarly, by looking into the remainder of Taylor expansion for each label k, Eq. 30 can be proved straightforwardly. In fact, by using the 2nd-order Taylor series with K categories, we have that:

Preserving Differential Privacy in Adversarial Learning with Provable Robustness to Adversarial Examples

Represent the short summary for finding the corresponding scientific article (long).

This paper 1) characterizes functions representable by ReLU DNNs, 2) formally studies the benefit of depth in such architectures,  3) gives an algorithm to implement empirical risk minimization to global optimality for two layer ReLU nets.