Question: Represent the scientific article to find a short TLDR

Question: Generative adversarial networks (GANs) have been shown to provide an effective way to model complex distributions and have obtained impressive on various challenging tasks. However, typical GANs require fully-observed data during training. In this paper, we present a GAN-based framework for learning from complex, high-dimensional incomplete data. The proposed framework learns a complete data generator along with a mask generator that models the missing data distribution. We further demonstrate how to impute missing data by equipping our framework with an adversarially trained imputer. We evaluate the proposed framework using a series of experiments with several types of missing data processes under the missing completely at random assumption. Generative adversarial networks (GANs) BID0 provide a powerful modeling framework for learning complex high-dimensional distributions. Unlike likelihood-based methods, GANs are referred to as implicit probabilistic models BID8. They represent a probability distribution through a generator that learns to directly produce samples from the desired distribution. The generator is trained adversarially by optimizing a minimax objective together with a discriminator. In practice, GANs have been shown to be very successful in a range of applications including generating photorealistic images BID3. Other than generating samples, many downstream tasks require a good generative model, such as image inpainting BID9 BID15.Training GANs normally requires access to a large collection of fully-observed data. However, it is not always possible to obtain a large amount of fully-observed data. Missing data is well-known to be prevalent in many real-world application domains where different data cases might have different missing entries. This arbitrary missingness poses a significant challenge to many existing machine learning models. Following BID6, the generative process for incompletely observed data can be described as shown below where x ∈ R n is a complete data vector and m ∈ {0, 1} n is a binary mask 2 that determines which entries in x to reveal: DISPLAYFORM0 Let x obs denote the observed elements of x, and x mis denote the missing elements according to the mask m. In addition, let θ denote the unknown parameters of the data distribution, and φ denote the unknown parameters for the mask distribution, which are usually assumed to be independent of θ. In the standard maximum likelihood setting, the unknown parameters are estimated by maximizing the 1 Our implementation is available at https://github.com/steveli/misgan 2 The complementm is usually referred to as the missing data indicator in the literature.following marginal likelihood, integrating over the unknown missing data values:p(x obs, m) = p θ (x obs, x mis)p φ (m|x obs, x mis)dx mis. characterize the missing data mechanism p φ (m|x obs, x mis) in terms of independence relations between the complete data x = [x obs, x mis] and the masks m:• Missing completely at random (MCAR): p φ (m|x) = p φ (m),• Missing at random (MAR): p φ (m|x) = p φ (m|x obs),• Not missing at random (NMAR): m depends on x mis and possibly also x obs.Most work on incomplete data assumes MCAR or MAR since under these assumptions p(x obs, m) can be factorized into p θ (x obs)p φ (m|x obs). With such decoupling, the missing data mechanism can be ignored when learning the data generating model while yielding correct estimates for θ. When p θ (x) does not admit efficient marginalization over x mis, estimation of θ is usually performed by maximizing a variational lower bound, as shown below, using the EM algorithm or a more general approach BID6 ):log p θ (x obs) ≥ E q(xmis|xobs) [log p θ (x obs, x mis) − log q(x mis |x obs)].The primary contribution of this paper is the development of a GAN-based framework for learning high-dimensional data distributions in the presence of incomplete observations. Our framework introduces an auxiliary GAN for learning a mask distribution to model the missingness. The masks are used to "mask" generated complete data by filling the indicated missing entries with a constant value. The complete data generator is trained so that the ing masked data are indistinguishable from real incomplete data that are masked similarly. Our framework builds on the ideas of AmbientGAN . AmbientGAN modifies the discriminator of a GAN to distinguish corrupted real samples from corrupted generated samples under a range of corruption processes (or measurement processes). For images, examples of the measurement processes include random dropout, blur, block-patch, and so on. Missing data can be seen as a special type of corruption, except that we have access to the missing pattern in addition to the corrupted measurements. Moreover, AmbientGAN assumes the measurement process is known or parameterized only by a few parameters, which is not the case in general missing data problems. We provide empirical evidence that the proposed framework is able to effectively learn complex, highdimensional data distributions from highly incomplete data when the GAN generator incorporates suitable priors on the data generating process. We further show how the architecture can be used to generate high-quality imputations. In the missing data problem, we know exactly which entries in each data examples are missing. Therefore, we can represent an incomplete data case as a pair of a partially-observed data vector x ∈ R n and a corresponding mask m ∈ {0, 1} n that indicates which entries in x are observed: x d is observed if m d = 1 otherwise x d is missing and might contain an arbitrary value that we should ignore. With this representation, an incomplete dataset is denoted D = {(x i, m i)} i=1,...,N (we assume instances are i.i.d. samples). We choose this representation instead of x obs because it leads to a cleaner description of the proposed MisGAN framework. It also suggests how MisGAN can be implemented efficiently in practice as both x and m are fixed-length vectors. We begin by defining a masking operator f τ that fills in missing entries with a constant value τ: DISPLAYFORM0 wherem denotes the complement of m and denotes element-wise multiplication. Two key ideas underlie the MisGAN framework. First, in addition to the complete data generator, we explicitly model the missing data process using a mask generator. Since the masks in the incomplete dataset are fully observed, we can estimate their distribution using a standard GAN. Second, we train the complete data generator adversarially by masking its outputs using generated masks and f τ and comparing to real incomplete data that are similarly masked by f τ.Specifically, we use two generator-discriminator pairs (G m, D m) and (G x, D x) for the masks and data respectively. In this paper, we focus on the missing completely at random (MCAR) case, where the two generators are independent of each other and have their own noise distributions p z and p ε. We define the following two loss functions, one for the masks and the other for the data: DISPLAYFORM1 The losses above follow the Wasserstein GAN formulation BID1, although the proposed framework is compatible with many GAN variations BID0; BID1. We optimize the generators and the discriminators according to the following objectives: DISPLAYFORM2 DISPLAYFORM3 where F x, F m are defined such that D x, D m are both 1-Lipschitz for Wasserstein GANs BID1. Practically, we follow the common practice of alternating between a few steps of optimizing the discriminators and one step of optimizing the generators BID0 BID1 BID1. The coefficient α is introduced when optimizing the mask generator G m with the aim of minimizing a combination of L m and L x. Although in theory we could choose α = 0 to train G m and D m without using the data, we find that choosing a small value such as α = 0.2 improves performance. This encourages the generated masks to match the distribution of the real masks and the masked generated complete samples to match masked real data. The overall structure of MisGAN is illustrated in FIG0.Note that the data discriminator D x takes as input the masked samples as if the data are fullyobserved. This allows us to use any existing architecture designed for complete data to construct the data discriminator. There is no need to develop customized neural network modules for dealing with missing data. For example, D x can be a standard convolutional network for image applications. Note that the masks are binary-valued. Since discrete data generating processes have zero gradient almost everywhere, to carry out gradient-based training for GANs, we relax the output of the mask generator G m from {0, 1} n to n. We employ a sigmoid activation σ λ (x) = 1/(1 + exp(−x/λ)) with a low temperature 0 < λ < 1 to encourage saturation and make the output closer to zero or one. Finally, we note that the discriminator D x in MisGAN is unaware of which entries are missing in the masked input samples, and does not even need to know which value τ is used for masking. In the next section, we present a theoretical analysis providing support for the idea that this type of masking process does not necessarily make it more difficult to recover the complete data distribution. The experiments provide compelling empirical evidence for the effectiveness of the proposed framework. In Section 2 we described how the discriminator D x in MisGAN takes as input the masked samples using without knowing what value τ is used or which entries in the input vector are missing. In this section, we discuss the following two important questions: i) Does the choice of the filled-in value τ affect the ability to recover the data distribution? ii) Does information about the location of missing values affect the ability to recover the data distribution?We address these questions in a simplified scenario where each dimension of the data vector takes values from a finite set P. For n-dimensional data, let M = {0, 1}n be the set of all possible masks and I = P n be the set of all possible data vectors. Also let D M and D I be the set of all possible probability distributions on M and I respectively, whose elements are non-negative and sum to one. We first discuss the case where the filled-in value τ is chosen from P.Given τ ∈ P and q ∈ D M, we can construct a left transition matrix T q,τ ∈ R I×I defined below where the (t, s)-th entry specifies the transition probability from a data vector s ∈ I to an outcome t ∈ I masked by f τ, which involves all possible masks under which s is converted into t by filling in the indicated missing entries with τ: DISPLAYFORM0 Let p * x ∈ D I be the unknown true data distribution we want to estimate. In the presence of missing data specified by q, the masked samples then follow the distribution p y = T q,τ p * x. Without imposing extra application-specific constraints, MisGAN with a fixed mask generator can be viewed as solving the linear system p y = T q,τ p x, where p x ∈ D I is the unknown data distribution to solve for. Here we assume that p y and T q,τ are given, as those can be estimated separately from a collection of fully-observed masks and masked samples. Note that a transition matrix preserves the sum of the vectors it is applied to since 1 T q,τ = 1. For p x to be a valid distribution vector, we only need the non-negativity constraint because any solution p x automatically sums to one. That is, estimating the data generating process in the presence of missing data based on the masking scheme used in MisGAN is equivalent to solving the linear system DISPLAYFORM1 In Theorem 1, we state a key property of the transition matrix T q,τ that leads to the answer to our questions. The proof of Theorem 1 is in Appendix A. Theorem 1. Given q ∈ D M, all transition matrices T q,τ with τ ∈ P have the same null space. Theorem 1 implies that if the solution to the constrained linear system FORMULA7 is not unique for a given τ 0 ∈ P, that is, there exists some non-negative DISPLAYFORM2 In other words, we have the following corollary: Corollary 1. Whether the true data distribution is uniquely recoverable is independent of the choice of the filled-in value τ.Here we only discuss the case when the probability of observing all features q is zero, where q denotes the scalar entry of q indexed by 1 ∈ M. Otherwise, the linear system is uniquely solvable as the transition matrix T q,τ0 has full rank. With the non-negativity constraint, it is possible that the solution for the linear system FORMULA7 is unique when the true data distribution p * x is sparse. Specifically, if there exists two indices s 1, s 2 ∈ I such that p * x (s 1) = p * x (s 2) = 0 and also v(s 1) > 0 and v(s 2) < 0 for all v ∈ Null(T q,τ) \ {0}, then the solution to is unique. Sparsity of the data distribution is a reasonable assumption in many situations. For example, natural images are typically considered to lie on a low dimensional manifold, which means most of the instances in I should have almost zero probability. On the other hand, when the missing rate is high, that is, if the masks in M that have many zeros are more probable, the null space of T q,τ will be larger and therefore it is more likely that the non-negative solution is not unique. proposed a sufficient condition on the sparsity of the non-negative solutions to a general underdetermined linear system that guarantees unique optimality. Next we note that in the case of τ ∈ P, an entry with value τ in a masked sample t ∈ I may come either from an observed entry with value τ in the unmasked sample or from an unobserved entry through the masking operation in. One might wonder if this prevents an algorithm from recovering the true distribution when it is otherwise possible to do so. In other words, if we take the location of the missing values into account, would that make the missing data problem less ill-posed? However, this is not the case, as we state in Corollary 2. The proof is in Appendix B where we discuss the case of τ / ∈ P. Corollary 2. If the linear system T q,τ p x = T q,τ p * x does not have a unique non-negative solution, then for this missing data problem, we cannot uniquely recover the true data distribution even if we take the location of the missing values into account. Note that the analysis in this section characterizes how difficult the missing data problem is, which is independent of the choice of the algorithm that solves it. In practice, it is useful to incorporate application-specific prior knowledge into the model to regularize the problem when it is ill-posed. For example, for modeling natural images, convolutional networks are commonly used to exploit the local structure of the data. In addition, decoder-based deep generative models such as GANs implicitly enforce some sparsity constraints due to the use of low dimensional latent codes in the generator, which also helps to regularize the problem. Finally, the following theorem justifies the training objective of MisGAN for the missing data problem (see Appendix A for details). Theorem 2. Given a mask distribution p φ (m), two distributions p θ (x) and p θ (x) induce the same distribution for f τ (x, m) if and only if they have the same marginals p θ (x obs |m) = p θ (x obs |m) for all masks m with p φ (m) > 0. Missing data imputation is an important task when dealing with incomplete data. In this section, we show how to impute missing data according to p(x mis |x obs) by equipping MisGAN with an imputer G i accompanied by a corresponding discriminator D i. The imputer is a function of the incomplete example (x, m) and a random vector ω drawn from a noise distribution p ω. It outputs the completed sample with the observed part in x kept intact. To train the imputer-equipped MisGAN, we define the loss for the imputer in addition to and: DISPLAYFORM3 We jointly learn the data generating process and the imputer according to the following objectives: DISPLAYFORM4 DISPLAYFORM5 DISPLAYFORM6 where we use β = 0.1 in the experiments when optimizing G x. This encourages the generated complete data to match the distribution of the imputed real data in addition to having the masked generated data match the masked real data. The overall structure for MisGAN imputation is illustrated in Figure 2.We can also train a stand-alone imputer using only with a pre-trained data generator G x. The architecture is as shown in Figure 2 with the faded parts removed. Moreover, it is also possible to train the imputer to target a different missing distribution p m with a pre-trained data generator G x alone without access to the original (incomplete) training data: DISPLAYFORM7 We construct the imputer G i (x, m, ω) as follows: DISPLAYFORM8 Figure 2: Architecture for MisGAN imputation. The complete data generator G x and the imputer G i can be trained jointly with all the components. We can also independently train the imputer G i without the faded parts if the data generator G x has been pre-trained.where G i generates the imputed with the same dimensionality as its input, x m + ω m, which could be implemented by a deep neural network. The masking outside of G i ensures that the observed part of x stays the same in the output of the imputer G i. The similar masking on the input of G i, x m + ω m, ensures that the amount of noise injected to G i scales with the number of missing dimensions. This is intuitive in the sense that when a data case is almost fully-observed, we expect less variety in p(x mis |x obs) and vice versa. Note that the noise ω needs to have the same dimensionality as x. In this section, we first assess various properties of MisGAN on the MNIST dataset: we demonstrate qualitatively how MisGAN behaves under different missing patterns and different architectures. We then conduct an ablation study to justify the construction of MisGAN. Finally, we compare MisGAN with various baseline methods on the missing data imputation task over three datasets under a series of missingness settings. Data We evaluate MisGAN on three datasets: MNIST, CIFAR-10 and CelebA. MNIST is a dataset of handwritten digits images of size 28×28 BID5. We use the provided 60,000 training examples for the experiments. CIFAR-10 is a dataset of 32×32 color images from 10 classes BID4. Similarly, we use 50,000 training examples for the experiments. CelebA is a large-scale face attributes dataset BID7 that contains 202,599 face images, where we use the provided aligned and cropped images and resize them to 64×64. For all three datasets, the range of pixel values of each image is rescaled to.Missing data distributions We consider three types of missing data distribution: i) Square observation: all pixels are missing except for a square occurring at a random location on the image. ii) Dropout: each pixel is independently missing according to a Bernoulli distribution. iii) Variablesize rectangular observation: all pixels are missing except for a rectangular observed region. The width and height of the rectangle are independently drawn from 25% to 75% of the image length uniformly at random, which in a 75% missing rate on average. In this missing data distribution, each example may have a different number of missing pixels. The highest per-example missing data rate under this mechanism is 93.75%.Evaluation metric We use the Fréchet Inception Distance (FID) BID2 to evaluate the quality of the learned generative model. For MNIST, instead of the Inception network trained on ImageNet BID13, we use a basic LeNet model 4 trained on the complete MNIST training set, and then take the 50-dimensional output from the second-to-last fully-connected layer as the features to compute the FID. For CIFAR-10 and CelebA, we follow the procedure described in BID2 to compute the FID using the pretrained Inception-v3 model. When evaluating generative models using the FID, we use the same number of generated samples as the size of the training set. In this section, we study various properties of MisGAN using the MNIST dataset. Architectures We consider two kinds of architecture for MisGAN: convolutional networks and fully connected networks. We follow the DCGAN architecture BID11 for (de)convolutional generators and discriminators to exploit the local structures of images. We call this model ConvMisGAN.To demonstrate the performance of MisGAN in the absence of the implicit structural regularization provided by the use of a convolutional network, we construct another MisGAN with only fullyconnected layers for both the generators and the discriminators, which we call FC-MisGAN.In the experiments, both Conv-MisGAN and FC-MisGAN are trained using the improved procedure for the Wasserstein GAN with gradient penalty BID1 ). Throughout we use τ = 0 for the masking operator and the temperature λ = 0.66 for the mask activation σ λ (x) described in Section 2.Baseline We compare MisGAN to a baseline model that is capable of learning from large-scale incomplete data: the generative convolutional arithmetic circuit (ConvAC) BID14. ConvAC is an expressive mixture model similar to sum-product networks BID10 with a compositional structure similar to deep convolutional networks. Most importantly, ConvAC admits tractable marginalization due to the product form of the base distributions for the mixtures, which makes it readily capable of learning with missing data. Results Figures 3 and 4 show the generated data samples as well as the learned mask samples produced by Conv-MisGAN and FC-MisGAN under the square observation and independent dropout missing mechanisms. From these , we can see that Conv-MisGAN produces visually better samples than FC-MisGAN on this problem. On the other hand, under the same missing rate, independent dropout leads to worse samples than square observations. Samples generated by ConvAC are shown in FIG0 in Appendix G.We quantitatively evaluate Conv-MisGAN, FC-MisGAN and ConvAC under two missing patterns with missing rates from 10% to 90% with a step of 10%. Figure 5 shows that MisGAN in general outperforms ConvAC as ConvAC tends to generate samples with aliasing artifacts as shown in FIG0. It also shows that in the square observation case, Conv-MisGAN and FC-MisGAN have similar performance in terms of their FIDs. However, under independent dropout, the performance of FC-MisGAN degrades significantly as the missing rate increases compared to Conv-MisGAN. This is because independent dropout with high missing rate makes the problem more challenging as it induces less overlapping co-occurrence among pixels, which degrades the signal for understanding the overall structure. This is illustrated in Figure 6 where the observed pattern comes from one of four equally probable 14×14 square quadrants with no overlap. Clearly this missing data problem is ill-posed and we could never uniquely determine the correlation between pixels across different quadrants without additional assumptions. The samples generated by the FC-MisGAN produce obvious discontinuity across the boundary of the quadrants as it does not rely on any prior knowledge about how pixels are correlated. The discontinuity artifact is less severe with Conv-MisGAN since the convolutional layers encourage local smoothness. This shows the importance of incorporating prior knowledge into the model when the problem is highly ill-posed. Ablation study We point out that the mask discriminator in MisGAN is important for learning the correct distribution robustly. Figure 7 shows two common failure scenarios that frequently happen with an AmbientGAN, which is essentially equivalent to a MisGAN without the mask discriminator. Figure 7 (left) shows a case where AmbientGAN produces perfectly consistent masked outputs, but the learned mask distribution is completely wrong. Since we use f τ =0 (x, m) = x m, it makes the role of x and m interchangeable when considering only the masked outputs. Even if we rescale the range of pixel values from to [−1, 1] to avoid this situation, AmbientGAN still fails often as shown in Figure 7 (right). In contrast, MisGAN avoids learning such degenerate solutions due to explicitly modeling the mask distribution. Missing data imputation We construct the imputer network G i defined in using a three-layer fully-connected network with 500 hidden units in the middle layers. imputation on different examples applying novel masks randomly drawn according to the same distribution. FIG5 (right) shows the imputation where each row corresponds to the same incomplete input. It demonstrates that the imputer can produce a variety of different imputed due to the random noise input to the imputer. We also note that if we modify to train the imputer together with the data generator from scratch without the mask generator/discriminator, it fails most of the time for a similar reason to why AmbientGAN fails. The learning problem is highly ill-posed without the agreement on the mask distribution. In this section, we quantitatively evaluate the performance of MisGAN on three datasets: MNIST, CIFAR-10, and CelebA. We focus on evaluating MisGAN on the missing data imputation task as it is widely studied and many baseline methods are available. Baselines We compare the MisGAN imputer to a range of baseline methods including the basic zero/mean imputation, matrix factorization, and the recently proposed Generative Adversarial Imputation Network (GAIN) BID16. GAIN is an imputation model that employs an imputer network to complete the missing data. It is trained adversarially with a discriminator that determines which entries in the completed data were actually observed and which were imputed. It has shown to outperform many state-of-the-art imputation methods. We impute all of the incomplete examples in the training set and use the FID between the imputed data and the original fully-observed data as the evaluation metric. Architecture We use convolutional generators and discriminators for MisGAN for all experiments in this section. For MNIST, we use the same fully-connected imputer network as described in the previous section; for CIFAR-10 and CelebA, we use a five-layer U-Net architecture BID12 for the imputer network G i in MisGAN.Results We compare all the methods under two missing patterns, square observation and independent dropout, with missing rates from 10% to 90%. FIG6 shows that MisGAN consistently outperforms other methods in all cases, especially under high missing rates. In our experiments, we found GAIN training to be quite unstable for the block missingness. We also observed that there is a "sweet spot" for the number of training epochs when training GAIN. If trained longer, the imputation behavior will gradually become similar to constant imputation (see Appendix H for details). On the other hand, we find that training MisGAN is more stable than training GAIN across all scenarios in the experiments. The imputation of MisGAN and GAIN are shown in Appendix E, F, and H. This work presents and evaluates a highly flexible framework for learning standard GAN data generators in the presence of missing data. Although we only focus on the MCAR case in this work, MisGAN can be easily extended to cases where the output of the data generator is provided to the mask generator. These modifications can capture both MAR and NMAR mechanisms. The question of learnability requires further investigation as the analysis in Section 3 no longer holds due to dependence between the transition matrix and the data distribution under MAR and NMAR. We have tried this modified architecture in our experiments and it showed similar as to the original MisGAN. This suggests that the extra dependencies may not adversely affect learnability. We leave the formal evaluation of this modified framework for future work. A PROOF OF THEOREM 1 AND THEOREM 2Let P be the finite set of feature values. For the n-dimensional case, let M = {0, 1} n be the set of masks and I = P n be the set of all possible feature vectors. Also let D M be the set of probability distributions on M, which implies m 0 and v∈I m(v) = 1 for all m ∈ M, where m(v) denotes the entry of m indexed by v. Given τ ∈ P and q ∈ D M, define the transformation DISPLAYFORM0 where is the entry-wise multiplication and 1{·} is the indicator function. Given m ∈ M, define an equivalent relation ∼ m on I by v ∼ m u iff v m = u m, and denote by [v] m the equivalence class containing v. Given q ∈ D M, let S q ⊂ M be the support of q, that is, DISPLAYFORM1 Given τ ∈ P and v ∈ I, let M τ,v denote the set of masks consistent with v in the sense that q(m) > 0 and v m = τm, that is, DISPLAYFORM2 Proof. This is clear from the following equation DISPLAYFORM3 which can be obtained from as follows, DISPLAYFORM4 Proposition 2. For any τ ∈ P, q ∈ D M and x ∈ R I, the vector T q,τ x determines the collection of marginals {x ([v] DISPLAYFORM5 Proof. Fix τ ∈ P, q ∈ D M and x ∈ R I. Since v m + τm ∈ [v] m, it suffices to show that we can solve for x ([v] m ) in terms of T q,τ x for m ∈ M τ,v = ∅. We use induction on the size of M τ,v.First consider the base case |M τ,v | = 1. Consider v 0 ∈ I with M τ,v0 = {m 0}. By FORMULA0, DISPLAYFORM6, which proves the base case. Now assume we can solve for x ([v] m ) in terms of T q,τ x for m ∈ S q and v ∈ I with |M τ,v | ≤ k. Consider v 0 ∈ I with |M τ,v0 | = k + 1; if no such v 0 exists, the holds trivially. Let M τ,v0 = {m 0, m 1, . . ., m k}. We need to show that T q,τ x determines x([v 0] m ) for = 0, 1,..., k. By again, DISPLAYFORM7 Let m = k =0 m, which may or may not belong to S q. Note that DISPLAYFORM8 and hence DISPLAYFORM9 Plugging FORMULA0 into FORMULA0 yields DISPLAYFORM10 Note that DISPLAYFORM11 It follows from FORMULA0 and FORMULA0 Theorem 1 is a direct consequence of Proposition 1 and Proposition 2 as the collection of marginals {x ([v] m ): v ∈ I, m ∈ S q } is independent of τ. Therefore, if x 1, x 2 ∈ R I satisfy T q,τ0 x 1 = T q,τ0 x 2 for some τ 0 ∈ P, then T q,τ x 1 = T q,τ x 2 for all τ ∈ P. Theorem 1 is a special case when x 1 = 0.Moreover, Proposition 2 also shows that MisGAN overall learns the distribution p(x obs, m), as x([v] m ) is equivalent to p(x obs |m) and T q,τ x is essentially the distribution of f τ (x, m) under the optimally learned missingness q = p(m). Theorem 2 basically restates Proposition 1 and Proposition 2. This is also true when τ / ∈ P according to Appendix B. Corollary 2 can be shown by augmenting the set of feature values by P = P ∪ {ψ} with a novel symbol ψ / ∈ P. If we choose τ = ψ for the masking operator, whenever we spot a ψ in a masked sample, we know that it corresponds to a missing entry. We can also construct the corresponding transition matrix T q,ψ ∈ R I ×I where I = (P) n given the mask distribution q ∈ D M before. In this setting, the generative model for missing data is equivalent to solving the linear system T q,ψ p x = T q,ψ p * x so that p x ∈ R I is non-negative and p x (s) = 0 for all s ∈ I \ I, where the true distribution p * x is given by p * x (s) = p * x (s) for all s ∈ I and zeros elsewhere. Theorem 1 implies that if the solution to original problem FORMULA7 is not unique, the non-negative solution to the augmented linear system with the extra constraint on I \ I with τ = ψ is not unique either. Root mean square error (RMSE) is a commonly used metric for evaluating the performance of missing data imputation, which computes the RMSE of the imputed missing values against the ground truth. However, in a complex system, the conditional distribution p(x mis |x obs) is likely to be highly multimodal. It's not guaranteed that the ground truth of the missing values in the incomplete dataset created under the missing completely at random (MCAR) assumption correspond to the global mode of p(x mis |x obs). A good imputation model might produce samples from p(x mis |x obs) associated with a higher density than the ground truth (or from other modes that are similarly probable). In this case, it will lead to a large error in terms of metrics like RMSE as multiple modes might be far away from each other in a complex distribution. Therefore, we instead compute the FID between the distribution of the completed data and the distribution of the originally fully-observed data as our evaluation metric. This provides a practical way to assess how close a model imputes according to p(x mis |x obs) by comparing two groups of samples collectively. As a concrete example, FIG0 compares the two evaluation metrics on MNIST, our distributionbased FID and the ground truth-based RMSE. It shows that the rankings on most of the missing rates are not consistent across the two metrics. In particular, under 90% missing rate, MisGAN is worse than GAIN and matrix factorization in terms of RMSE, but significantly better in terms of FID. FIG0 plots the imputation of the three methods mentioned above. We can clearly see that MisGAN produces the best completion even though its RMSE is much higher than the other two. It's not surprising as the mean of p(x mis |x obs) minimizes the squared error in expectation, even if the mean might have low density. This probably explains why the blurry completion produced by matrix factorization achieve the lowest RMSE. All of the generators and discriminators in Conv-MisGAN follow the architecture used by the DCGAN model BID11 with 128-dimensional latent code. As For the imputer network for MisGAN trained on CIFAR-10 and CelebA, we follow the U-Net implementation of the CycleGAN and pix2pix work 6. In the experiments, we use 5-layer U-Nets for both CIFAR-10 and CelebA.For training Wasserstein GAN with gradient penalty, We use all the default hyperparameters reported in BID1. For all the datasets, MisGAN is trained for 300 epochs. We train MisGAN imputer for 1000 epochs for MNIST and CIFAR-10 as the networks are smaller and 600 epochs for CelebA.For ConvAC, we use the same architecture described in BID14. We train ConvAC for 1000 epochs using Adam optimizer with learning rate 10 G OF CONVAC FIG0 shows the samples generated by ConvAC trained with the square observation missing pattern on MNIST.H MISSING DATA IMPUTATION WITH GAIN FIG0 shows the imputation of GAIN on different epochs during training with the 20×20 square observation missingnss. We found that this is a common phenomenon for the square observation missing pattern. To obtain better for GAIN, we analyze the FIDs during the course of training and use the model that achieves the best FID to favorably compare with MisGAN for the square observation case. For CIFAR-10, we use the from the 500th epoch; for CelebA, we use the from the 50th epoch. Otherwise, we train GAIN for 1000 epochs for CIFAR-10 and 300 epochs for CelebA. Our implementation is adapted from the code released by the authors of GAIN.

Represent the TLDR for finding the corresponding scientific article

This paper presents a GAN-based framework for learning the distribution from high-dimensional incomplete data.

represent the tldr for finding the corresponding scientific article.

------

For instance, <<A simple and effective alternative to adversarial imitation learning: initialize experience replay buffer with demonstrations, set their reward to +1, set reward for all other data to 0, run Q-learning or soft actor-critic to train.>>  to <<Learning to imitate expert behavior from demonstrations can be challenging, especially in environments with high-dimensional, continuous observations and unknown dynamics. Supervised learning methods based on behavioral cloning (BC) suffer from distribution shift: because the agent greedily imitates demonstrated actions, it can drift away from demonstrated states due to error accumulation. Recent methods based on reinforcement learning (RL), such as inverse RL and generative adversarial imitation learning (GAIL), overcome this issue by training an RL agent to match the demonstrations over a long horizon. Since the true reward function for the task is unknown, these methods learn a reward function from the demonstrations, often using complex and brittle approximation techniques that involve adversarial training. We propose a simple alternative that still uses RL, but does not require learning a reward function. The key idea is to provide the agent with an incentive to match the demonstrations over a long horizon, by encouraging it to return to demonstrated states upon encountering new, out-of-distribution states. We accomplish this by giving the agent a constant reward of r=+1 for matching the demonstrated action in a demonstrated state, and a constant reward of r=0 for all other behavior. Our method, which we call soft Q imitation learning (SQIL), can be implemented with a handful of minor modifications to any standard Q-learning or off-policy actor-critic algorithm. Theoretically, we show that SQIL can be interpreted as a regularized variant of BC that uses a sparsity prior to encourage long-horizon imitation. Empirically, we show that SQIL outperforms BC and achieves competitive compared to GAIL, on a variety of image-based and low-dimensional tasks in Box2D, Atari, and MuJoCo. This paper is a proof of concept that illustrates how a simple imitation method based on RL with constant rewards can be as effective as more complex methods that use learned rewards. Many sequential decision-making problems can be tackled by imitation learning: an expert demonstrates near-optimal behavior to an agent, and the agent attempts to replicate that behavior in novel situations . This paper considers the problem of training an agent to imitate an expert, given expert action demonstrations and the ability to interact with the environment. The agent does not observe a reward signal or query the expert, and does not know the state transition dynamics. Standard approaches based on behavioral cloning (BC) use supervised learning to greedily imitate demonstrated actions, without reasoning about the consequences of actions . As a , compounding errors cause the agent to drift away from the demonstrated states . The problem with BC is that, when the agent drifts and encounters out-of-distribution states, the agent does not know how to return to the demonstrated states. Recent methods based on inverse reinforcement learning (IRL) overcome this issue by training an RL agent not only to imitate demonstrated actions, but also to visit demonstrated states (; ; b;). This is also the core idea behind generative adversarial imitation learning (GAIL) , which implements IRL using generative adversarial networks (; a). Since the true reward function for the task is unknown, these methods construct a reward signal from the demonstrations through adversarial training, making them difficult to implement and use in practice . The main idea in this paper is that the effectiveness of adversarial imitation methods can be achieved by a much simpler approach that does not require adversarial training, or indeed learning a reward function at all. Intuitively, adversarial methods encourage long-horizon imitation by providing the agent with an incentive to imitate the demonstrated actions in demonstrated states, and an incentive to take actions that lead it back to demonstrated states when it encounters new, out-ofdistribution states. One of the reasons why adversarial methods outperform greedy methods, such as BC, is that greedy methods only do, while adversarial methods do both and. Our approach is intended to do both and without adversarial training, by using constant rewards instead of learned rewards. The key idea is that, instead of using a learned reward function to provide a reward signal to the agent, we can simply give the agent a constant reward of r = +1 for matching the demonstrated action in a demonstrated state, and a constant reward of r = 0 for all other behavior. We motivate this approach theoretically, by showing that it implements a regularized variant of BC that learns long-horizon imitation by (a) imposing a sparsity prior on the reward function implied by the imitation policy, and (b) incorporating information about the state transition dynamics into the imitation policy. Intuitively, our method accomplishes (a) by training the agent using an extremely sparse reward function -+1 for demonstrations, 0 everywhere else -and accomplishes (b) by training the agent with RL instead of supervised learning. We instantiate our approach with soft Q-learning by initializing the agent's experience replay buffer with expert demonstrations, setting the rewards to a constant r = +1 in the demonstration experiences, and setting rewards to a constant r = 0 in all of the new experiences the agent collects while interacting with the environment. Since soft Q-learning is an off-policy algorithm, the agent does not necessarily have to visit the demonstrated states in order to experience positive rewards. Instead, the agent replays the demonstrations that were initially added to its buffer. Thus, our method can be applied in environments with stochastic dynamics and continuous states, where the demonstrated states are not necessarily reachable by the agent. We call this method soft Q imitation learning (SQIL). The main contribution of this paper is SQIL: a simple and general imitation learning algorithm that is effective in MDPs with high-dimensional, continuous observations and unknown dynamics. We run experiments in four image-based environments -Car Racing, Pong, Breakout, and Space Invadersand three low-dimensional environments -Humanoid, HalfCheetah, and Lunar Lander -to compare SQIL to two prior methods: BC and GAIL. We find that SQIL outperforms BC and achieves competitive compared to GAIL. Our experiments illustrate two key benefits of SQIL: that it can overcome the state distribution shift problem of BC without adversarial training or learning a reward function, which makes it easier to use, e.g., with images, and that it is simple to implement using existing Q-learning or off-policy actor-critic algorithms. SQIL performs soft Q-learning with three small, but important, modifications: it initially fills the agent's experience replay buffer with demonstrations, where the rewards are set to a constant r = +1; as the agent interacts with the world and accumulates new experiences, it adds them to the replay buffer, and sets the rewards for these new experiences to a constant r = 0; and it balances the number of demonstration experiences and new experiences (50% each) in each sample from the replay buffer. 1 These three modifications are motivated theoretically in Section 3, via an equivalence to a regularized variant of BC. Intuitively, these modifications create a simple reward structure that gives the agent an incentive to imitate the expert in demonstrated states, and to take actions that lead it back to demonstrated states when it strays from the demonstrations. Algorithm 1 Soft Q Imitation Learning (SQIL) 1: Require λsamp ∈ R ≥0, Ddemo 2: Initialize Dsamp ← ∅ 3: while Q θ not converged do 4: Sample transition (s, a, s) with imitation policy π(a|s) ∝ exp (Q θ (s, a)) 6: Dsamp ← Dsamp ∪ {(s, a, s)} 7: end while Crucially, since soft Q-learning is an off-policy algorithm, the agent does not necessarily have to visit the demonstrated states in order to experience positive rewards. Instead, the agent replays the demonstrations that were initially added to its buffer. Thus, SQIL can be used in stochastic environments with high-dimensional, continuous states, where the demonstrated states may never actually be encountered by the agent. SQIL is summarized in Algorithm 1, where Q θ is the soft Q function, D demo are demonstrations, δ 2 is the squared soft Bellman error, and r ∈ {0, 1} is a constant reward. 2 The experiments in Section 4 use a convolutional neural network or multi-layer perceptron to model Q θ, where θ are the weights of the neural network. Section A.3 in the appendix contains additional implementation details, including values for the hyperparameter λ samp; note that the simple default value of λ samp = 1 works well across a variety of environments. As the imitation policy in line 5 of Algorithm 1 learns to behave more like the expert, a growing number of expert-like transitions get added to the buffer D samp with an assigned reward of zero. This causes the effective reward for mimicking the expert to decay over time. Balancing the number of demonstration experiences and new experiences (50% each) sampled for the gradient step in line 4 ensures that this effective reward remains at least 1/(1 + λ samp), instead of decaying to zero. In practice, we find that this reward decay does not degrade performance if SQIL is halted once the squared soft Bellman error loss converges to a minimum (e.g., see Figure 8 in the appendix). Note that prior methods also require similar techniques: both GAIL and adversarial IRL (AIRL) balance the number of positive and negative examples in the training set of the discriminator, and AIRL tends to require early stopping to avoid overfitting. To understand why SQIL works, we sketch a surprising theoretical : SQIL is equivalent to a variant of behavioral cloning (BC) that uses regularization to overcome state distribution shift. BC is a simple approach that seeks to imitate the expert's actions using supervised learning -in particular, greedily maximizing the conditional likelihood of the demonstrated actions given the demonstrated states, without reasoning about the consequences of actions. Thus, when the agent makes small mistakes and enters states that are slightly different from those in the demonstrations, the distribution mismatch between the states in the demonstrations and those actually encountered by the agent leads to compounding errors . We show that SQIL is equivalent to augmenting BC with a regularization term that incorporates information about the state transition dynamics into the imitation policy, and thus enables long-horizon imitation. Maximum entropy model of expert behavior. SQIL is built on soft Q-learning, which assumes that expert behavior follows the maximum entropy model . In an infinite-horizon Markov Decision Process (MDP) with a continuous state space S and discrete action space A, 3 the expert is assumed to follow a policy π that maximizes reward R(s, a). The policy π forms a Boltzmann distribution over actions, where Q is the soft Q function. The soft Q values are a function of the rewards and dynamics, given by the soft Bellman equation, In our imitation setting, the rewards and dynamics are unknown. The expert generates a fixed set of demonstrations D demo, by rolling out their policy π in the environment and generating state transitions (s, a, s) ∈ D demo. Training an imitation policy with standard BC corresponds to fitting a parametric model π θ that minimizes the negative log-likelihood loss, In our setting, instead of explicitly modeling the policy π θ, we can represent the policy π in terms of a soft Q function Q θ via Equation 2: Using this representation of the policy, we can train Q θ via the maximum-likelihood objective in Equation 4: However, optimizing the BC loss in Equation 6 does not in general yield a valid soft Q function Q θ -i.e., a soft Q function that satisfies the soft Bellman equation (Equation 3) with respect to the dynamics and some reward function. The problem is that the BC loss does not incorporate any information about the dynamics into the learning objective, so Q θ learns to greedily assign high values to demonstrated actions, without considering the state transitions that occur as a consequence of actions. As a , Q θ may output arbitrary values in states that are off-distribution from the demonstrations D demo. In Section 3.2, we describe a regularized BC algorithm that adds constraints to ensure that Q θ is a valid soft Q function with respect to some implicitly-represented reward function, and further regularizes the implicit rewards with a sparsity prior. In Section 3.3, we show that this approach recovers an algorithm similar to SQIL. Under the maximum entropy model described in Section 3.1, expert behavior is driven by a reward function, a soft Q function that computes expected future returns, and a policy that takes actions with high soft Q values. In the previous section, we used these assumptions to represent the imitation policy in terms of a model of the soft Q function Q θ (Equation 5). In this section, we represent the reward function implicitly in terms of Q θ, as shown in Equation 7. This allows us to derive SQIL as a variant of BC that imposes a sparsity prior on the implicitly-represented rewards. Sparsity regularization. The issue with BC is that, when the agent encounters states that are outof-distribution with respect to D demo, Q θ may output arbitrary values. One solution from prior work is to regularize Q θ with a sparsity prior on the implied rewards -in particular, a penalty on the magnitude of the rewards s∈S,a∈A |R q (s, a)| implied by Q θ via the soft Bellman equation (Equation 3), where Note that the reward function R q is not explicitly modeled in this method. Instead, we directly minimize the magnitude of the right-hand side of Equation 7, which is equivalent to minimizing |R q (s, a)|. The purpose of the penalty on |R q (s, a)| is two-fold: it imposes a sparsity prior motivated by prior work , and it incorporates information about the state transition dynamics into the imitation learning objective, since R q (s, a) is a function of an expectation over next state s. is critical for learning long-horizon behavior that imitates the demonstrations, instead of greedy maximization of the action likelihoods in standard BC. For details, see. Approximations for continuous states. Unlike the discrete environments tested in , we assume the continuous state space S cannot be enumerated. Hence, we approximate the penalty s∈S,a∈A |R q (s, a)| by estimating it from samples: transitions (s, a, s) observed in the demonstrations D demo, as well as additional rollouts D samp periodically sampled during training using the latest imitation policy. This approximation, which follows the standard approach to constraint sampling , ensures that the penalty covers the state distribution actually encountered by the agent, instead of only the demonstrations. To make the penalty continuously differentiable, we introduce an additional approximation: instead of penalizing the absolute value |R q (s, a)|, we penalize the squared value (R q (s, a)) 2. Note that since the reward function R q is not explicitly modeled, but instead defined via Q θ in Equation 7, the squared penalty (R q (s, a)) 2 is equivalent to the squared soft Bellman error Regularized BC algorithm. Formally, we define the regularized BC loss function adapted from as where λ ∈ R ≥0 is a constant hyperparameter, and δ 2 denotes the squared soft Bellman error defined in Equation 1. The BC loss encourages Q θ to output high values for demonstrated actions at demonstrated states, and the penalty term propagates those high values to nearby states. In other words, Q θ outputs high values for actions that lead to states from which the demonstrated states are reachable. Hence, when the agent finds itself far from the demonstrated states, it takes actions that lead it back to the demonstrated states. The RBC algorithm follows the same procedure as Algorithm 1, except that in line 4, RBC takes a gradient step on the RBC loss from Equation 8 instead of the SQIL loss. The gradient of the RBC loss in Equation 8 is proportional to the gradient of the SQIL loss in line 4 of Algorithm 1, plus an additional term that penalizes the soft value of the initial state s 0 (full derivation in Section A.1 of the appendix): In other words, SQIL solves a similar optimization problem to RBC. The reward function in SQIL also has a clear connection to the sparsity prior in RBC: SQIL imposes the sparsity prior from RBC, by training the agent with an extremely sparse reward function -r = +1 at the demonstrations, and r = 0 everywhere else. Thus, SQIL can be motivated as a practical way to implement the ideas for regularizing BC proposed in. The main benefit of using SQIL instead of RBC is that SQIL is trivial to implement, since it only requires a few small changes to any standard Q-learning implementation (see Section 2). Extending SQIL to MDPs with a continuous action space is also easy, since we can simply replace Q-learning with an off-policy actor-critic method (see Section 4.3). Given the difficulty of implementing deep RL algorithms correctly , this flexibility makes SQIL more practical to use, since it can be built on top of existing implementations of deep RL algorithms. Furthermore, the ablation study in Section 4.4 suggests that SQIL actually performs better than RBC. Our experiments aim to compare SQIL to existing imitation learning methods on a variety of tasks with high-dimensional, continuous observations, such as images, and unknown dynamics. We benchmark SQIL against BC and GAIL 4 on four image-based games -Car Racing, Pong, Breakout, and Space Invaders -and three state-based tasks -Humanoid, HalfCheetah, and Lunar Lander (; ;). We also investigate which components of SQIL contribute most to its performance via an ablation study on the Lunar Lander game. Section A.3 in the appendix contains additional experimental details. The goal of this experiment is to study not only how well each method can mimic the expert demonstrations, but also how well they can acquire policies that generalize to new states that are not seen in the demonstrations. To do so, we train the imitation agents in an environment with a different initial state distribution S train 0 than that of the expert demonstrations S demo 0, allowing us to systematically control the mismatch between the distribution of states in the demonstrations and the states actually encountered by the agent. We run experiments on the Car Racing game from OpenAI Gym. To create S train 0, the car is rotated 90 degrees so that it begins perpendicular to the track, instead of parallel to the track as in S demo 0. This intervention presents a significant generalization challenge to the imitation learner, since the expert demonstrations do not contain any examples of states where the car is perpendicular to the road, or even significantly off the road axis. The agent must learn to make a tight turn to get back on the road, then stabilize its orientation so that it is parallel to the road, and only then proceed forward to mimic the expert demonstrations. The in Figure 1 show that SQIL and BC perform equally well when there is no variation in the initial state. The task is easy enough that even BC achieves a high reward. Note that, in the unperturbed condition (right column), BC substantially outperforms GAIL, despite the wellknown shortcomings of BC. This indicates that the adversarial optimization in GAIL can substantially hinder learning, even in settings where standard BC is sufficient. SQIL performs much better than BC when starting from S train 0, showing that SQIL is capable of generalizing to a new initial state distribution, while BC is not. SQIL learns to make a tight turn that takes the car through the grass and back onto the road, then stabilizes the car's orientation so that it is parallel to the track, and then proceeds forward like the expert does in the demonstrations. BC tends to drive straight ahead into the grass instead of turning back onto the road. 4 For all the image-based tasks, we implement a version of GAIL that uses deep Q-learning (GAIL-DQL) instead of TRPO as in the original GAIL paper , since Q-learning performs better than TRPO in these environments, and because this allows for a head-to-head comparison of SQIL and GAIL: both algorithms use the same underlying RL algorithm, but provide the agent with different rewards -SQIL provides constant rewards, while GAIL provides learned rewards. We use the standard GAIL-TRPO method as a baseline for all the low-dimensional tasks, since TRPO performs better than Q-learning in these environments. The original GAIL method implicitly encodes prior knowledge -namely, that terminating an episode is either always desirable or always undesirable. As pointed out in , this makes comparisons to alternative methods unfair. We implement the unbiased version of GAIL proposed by , and use this in all of the experiments. Comparisons to the biased version with implicit termination knowledge are included in Section A.2 in the appendix. SQIL outperforms GAIL in both conditions. Since SQIL and GAIL both use deep Q-learning for RL in this experiment, the gap between them may be attributed to the difference in the reward functions they use to train the agent. SQIL benefits from providing a constant reward that does not require fitting a discriminator, while GAIL struggles to train a discriminator to provide learned rewards directly from images. The in Figure 2 show that SQIL outperforms BC on Pong, Breakout, and Space Invaders -additional evidence that BC suffers from compounding errors, while SQIL does not. SQIL also outperforms GAIL on all three games, illustrating the difficulty of using GAIL to train an imagebased discriminator, as in Section 4.1. The experiments in the previous sections evaluate SQIL on MDPs with a discrete action space. This section illustrates how SQIL can be adapted to continuous actions. We instantiate SQIL using soft actor-critic (SAC) -an off-policy RL algorithm that can solve continuous control tasks . In particular, SAC is modified in the following ways: the agent's experience replay buffer is initially filled with expert demonstrations, where rewards are set to r = +1, when taking gradient steps to fit the agent's soft Q function, a balanced number of demonstration experiences and new experiences (50% each) are sampled from the replay buffer, and the agent observes rewards of r = 0 during its interactions with the environment, instead of an extrinsic reward signal that specifies the desired task. This instantiation of SQIL is compared to GAIL on the Humanoid (17 DoF) and HalfCheetah (6 DoF) tasks from MuJoCo. The show that SQIL outperforms BC and performs comparably to GAIL on both tasks, demonstrating that SQIL can be successfully deployed on problems with continuous actions, and that SQIL can perform well even with a small number of demonstrations. This experiment also illustrates how SQIL can be run on top of SAC or any other off-policy value-based RL algorithm. We hypothesize that SQIL works well because it combines information about the expert's policy from demonstrations with information about the environment dynamics from rollouts of the imitation policy periodically sampled during training. We also expect RBC to perform comparably to SQIL, since their objectives are similar. To test these hypotheses, we conduct an ablation study using the Lunar Lander game from OpenAI Gym. As in Section 4.1, we control the mismatch between the distribution of states in the demonstrations and the states encountered by the agent by manipulating the initial state distribution. To create S train 0, the agent is placed in a starting position never visited in the demonstrations. In the first variant of SQIL, λ samp is set to zero, to prevent SQIL from using additional samples drawn from the environment (see line 4 of Algorithm 1). This comparison tests if SQIL really needs to interact with the environment, or if it can rely solely on the demonstrations. In the second condition, γ is set to zero to prevent SQIL from accessing information about state transitions (see Equation 1 and line 4 of Algorithm 1). This comparison tests if SQIL is actually extracting information about the dynamics from the samples, or if it can perform just as well with a naïve regularizer (setting γ to zero effectively imposes a penalty on the L2-norm of the soft Q values instead of the squared soft Bellman error). In the third condition, a uniform random policy is used to sample additional rollouts, instead of the imitation policy π θ (see line 6 of Algorithm 1). This comparison tests how important it is that the samples cover the states encountered by the agent during training. In the fourth condition, we use RBC to optimize the loss in Equation 8 The in Figure 4 show that all methods perform well when there is no variation in the initial state. When the initial state is varied, SQIL performs significantly better than BC, GAIL, and the ablated variants of SQIL. This confirms our hypothesis that SQIL needs to sample from the environment using the imitation policy, and relies on information about the dynamics encoded in the samples. Surprisingly, SQIL outperforms RBC by a large margin, suggesting that the penalty on the soft value of the initial state V (s 0), which is present in RBC but not in SQIL (see Equation 9), degrades performance. Related work. Concurrently with SQIL, two other imitation learning algorithms that use constant rewards instead of a learned reward function were developed . We see our paper as contributing additional evidence to support this core idea, rather than proposing a competing method. First, SQIL is derived from sparsity-regularized BC, while the prior methods are derived from an alternative formulation of the IRL objective and from support estimation methods , showing that different theoretical approaches independently lead to using RL with constant rewards as an alternative to adversarial training -a sign that this idea may be a promising direction for future work. Second, SQIL is shown to outperform BC and GAIL in domains that were not evaluated in or -in particular, tasks with image observations and significant shift in the state distribution between the demonstrations and the training environment. Summary. We contribute the SQIL algorithm: a general method for learning to imitate an expert given action demonstrations and access to the environment. Simulation experiments on tasks with high-dimensional, continuous observations and unknown dynamics show that our method outperforms BC and achieves competitive compared to GAIL, while being simple to implement on top of existing off-policy RL algorithms. Limitations and future work. We have not yet proven that SQIL matches the expert's state occupancy measure in the limit of infinite demonstrations. One direction for future work would be to rigorously show whether or not SQIL has this property. Another direction would be to extend SQIL to recover not just the expert's policy, but also their reward function; e.g., by using a parameterized reward function to model rewards in the soft Bellman error terms, instead of using constant rewards. This could provide a simpler alternative to existing adversarial IRL algorithms. (s, a) ). Splitting up the squared soft Bellman error terms for D demo and D samp in Equation 8, Setting γ 1 turns the inner sum in the first term into a telescoping sum: Since s T is assumed to be absorbing, V (s T) is zero. Thus, In our experiments, we have that all the demonstration rollouts start at the same initial state s 0. Thus, where λ samp ∈ R ≥0 is a constant hyperparameter. As discussed in Section 4, to correct the original GAIL method's biased handling of rewards at absorbing states, we implement the suggested changes to GAIL in Section 4.2 of: adding a transition to an absorbing state and a self-loop at the absorbing state to the end of each rollout sampled from the environment, and adding a binary feature to the observations indicating whether or not a state is absorbing. This enables GAIL to learn a non-zero reward for absorbing states. We refer to the original, biased GAIL method as GAIL-DQL-B and GAIL-TRPO-B, and the unbiased version as GAIL-DQL-U and GAIL-TRPO-U. The mechanism for learning terminal rewards proposed in does not apply to SQIL, since SQIL does not learn a reward function. SQIL implicitly assumes a reward of zero at absorbing states in demonstrations. This is the case in all our experiments, which include some environments where terminating the episode is always undesirable (e.g., walking without falling down) and other environments where success requires terminating the episode (e.g., landing at a target), suggesting that SQIL is not sensitive to the choice of termination reward, and neither significantly benefits nor is significantly harmed by setting the termination reward to zero. Car Racing. The in Figure 5 show that both the biased (GAIL-DQL-B) and unbiased (GAIL-DQL-U) versions of GAIL perform equally poorly. The problem of training an image-based discriminator for this task may be difficult enough that even with an unfair bias toward avoiding crashes that terminate the episode, GAIL-DQL-B does not perform better than GAIL-DQL-U. Figure 6 show that SQIL outperforms both variants of GAIL on Pong and the unbiased version of GAIL (GAIL-DQL-U) on Breakout and Space Invaders, but performs comparably to the biased version of GAIL (GAIL-DQL-B) on Space Invaders and worse than it on Breakout. This may be due to the fact that in Breakout and Space Invaders, the agent has multiple livesfive in Breakout, and three in Space Invaders -and receives a termination signal that the episode has ended after losing each life. Thus, the agent experiences many more episode terminations than in Pong, exacerbating the bias in the way the original GAIL method handles rewards at absorbing states. Our implementation of GAIL-DQL-B in this experiment provides a learned reward of r(s, a) = − log (1 − D(s, a) ), where D is the discriminator (see Section A.3 in the appendix for details). The learned reward is always positive, while the implicit reward at an absorbing state is zero. Thus, the agent is inadvertently encouraged to avoid terminating the episode. For Breakout and Space Invaders, this just happens to be the right incentive, since the objective is to stay alive as long as possible. GAIL-DQL-B outperforms SQIL in Breakout and performs comparably to SQIL in Space Invaders because GAIL-DQL-B is accidentally biased in the right way. Lunar Lander. The in Figure 7 show that when the initial state is varied, SQIL outperforms the unbiased variant of GAIL (GAIL-TRPO-U), but underperforms against the biased version of GAIL (GAIL-TRPO-B). The latter is likely due to the fact that the implementation of GAIL-TRPO-B we used in this experiment provides a learned reward of r(s, a) = log (D (s, a) ), where D is the discriminator (see Section A.3 in the appendix for details). The learned reward is always negative, while the implicit reward at an absorbing state is zero. Thus, the agent is inadvertently encouraged to terminate the episode quickly. For the Lunar Lander game, this just happens to be the right incentive, since the objective is to land on the ground and thereby terminate the episode. As in the Atari experiments, GAIL-TRPO-B performs better than SQIL in this experiment because GAIL-TRPO-B is accidentally biased in the right way. To ensure fair comparisons, the same network architectures were used to evaluate SQIL, GAIL, and BC. For Lunar Lander, we used a network architecture with two fully-connected layers containing 128 hidden units each to represent the Q network in SQIL, the policy and discriminator networks in GAIL, and the policy network in BC. For Car Racing, we used four convolutional layers (following ) and two fully-connected layers containing 256 hidden units each. For Humanoid and HalfCheetah, we used two fully-connected layers containing 256 hidden units each. For Atari, we used the convolutional neural network described in to represent the Q network in SQIL, as well as the Q network and discriminator network in GAIL. To ensure fair comparisons, the same demonstration data were used to train SQIL, GAIL, and BC. For Lunar Lander, we collected 100 demonstration rollouts. For Car Racing, Pong, Breakout, and Space Invaders, we collected 20 demonstration rollouts. Expert demonstrations were generated from scratch for Lunar Lander using DQN , and collected from open-source pretrained policies for Car Racing as well as Humanoid and HalfCheetah . The Humanoid demonstrations were generated by a stochastic expert policy, while the HalfCheetah demonstrations were generated by a deterministic expert policy; both experts were trained using TRPO. 6 We used two open-source implementations of GAIL: for Lunar Lander, and for MuJoCo. We adapted the OpenAI Baselines implementation of GAIL to use soft Q-learning for Car Racing and Atari. Expert demonstrations were generated from scratch for Atari using DQN. For Lunar Lander, we set λ samp = 10 −6. For Car Racing, we set λ samp = 0.01. For all other environments, we set λ samp = 1. SQIL was not pre-trained in any of the experiments. GAIL was pre-trained using BC for HalfCheetah, but was not pre-trained in any other experiments. In standard implementations of soft Q-learning and SAC, the agent's experience replay buffer typically has a fixed size, and once the buffer is full, old experiences are deleted to make room for new experiences. In SQIL, we never delete demonstration experiences from the replay buffer, but otherwise follow the standard implementation. We use Adam to take the gradient step in line 4 of Algorithm 1. The BC and GAIL performance metrics in Section 4.3 are taken from . 7 The GAIL and SQIL policies in Section 4.3 are set to be deterministic during the evaluation rollouts used to measure performance.>>

We show that posterior collapse in linear VAEs is caused entirely by marginal log-likelihood (not ELBO). Experiments on deep VAEs suggest a similar phenomenon is at play.


Question: Represent the article for finding a single-sentence TLDR

Question: Due to a resource-constrained environment, network compression has become an important part of deep neural networks research. In this paper, we propose a new compression method, Inter-Layer Weight Prediction (ILWP) and quantization method which quantize the predicted residuals between the weights in all convolution layers based on an inter-frame prediction method in conventional video coding schemes. Furthermore, we found a phenomenon Smoothly Varying Weight Hypothesis (SVWH) which is that the weights in adjacent convolution layers share strong similarity in shapes and values, i.e., the weights tend to vary smoothly along with the layers. Based on SVWH, we propose a second ILWP and quantization method which quantize the predicted residuals between the weights in adjacent convolution layers. Since the predicted weight residuals tend to follow Laplace distributions with very low variance, the weight quantization can more effectively be applied, thus producing more zero weights and enhancing the weight compression ratio. In addition, we propose a new inter-layer loss for eliminating non-texture bits, which enabled us to more effectively store only texture bits. That is, the proposed loss regularizes the weights such that the collocated weights between the adjacent two layers have the same values. Finally, we propose an ILWP with an inter-layer loss and quantization method. Our comprehensive experiments show that the proposed method achieves a much higher weight compression rate at the same accuracy level compared with the previous quantization-based compression methods in deep neural networks. Deep neural networks have demonstrated great performance for various tasks in many fields, such as image classification (LeCun et al. 1990a; Krizhevsky et al. 2012; He et al. 2016), object detection (Ren et al. 2015; He et al. 2017; Redmon & Farhadi 2018), image captioning , and speech recognition Xiong et al. 2018). Wide and deep neural networks achieved great accuracy with the aid of the enormous number of weight parameters and high computational cost (Simonyan & Zisserman 2014; He et al. 2016;). However, as demands toward constructing the neural networks in the resource-constrained environments have been increasing, making the resource-efficient neural network while maintaining its accuracy becomes an important research area of deep neural networks. Several studies have aimed to solve this problem. In LeCun et al. (1990b), , Han et al. (2015b) and , network pruning methods were proposed for memory-efficient architecture, where unimportant weights were forced to have zero values in terms of accuracy. , and Han et al. (2015a), weights were quantized and stored in memory, enabling less memory usage of deep neural networks. On the other hand, some literature decomposed convolution operations into sub operations (e.g., depth-wise separable convolution) requiring less computation costs at similar accuracy levels (Howard et al. 2017; Sandler et al. 2018; Ma et al. 2018). In this paper, we show that the weights between the adjacent two convolution layers tend to share high similarity in shapes and values. We call this phenomenon Smoothly Varying Weight Hypothesis (SVWH). This paper explores an efficient neural network method that fully takes the advantage of SVWH. Specifically, inspired by the prediction techniques widely used in video compression field (Wiegand et al. 2003; Sullivan et al. 2012), we propose a new weight compression scheme based on an inter-layer weight prediction technique, which can be successfully incorporated into the depth-wise separable convolutional blocks (Howard et al. 2017; Sandler et al. 2018; Ma et al. 2018). Contributions: Main contributions of this paper are listed below: • From comprehensive experiments, we find out that the weights between the adjacent layers tend to share strong similarities, which lead us to establishing SVWH. • Based on SVWH, we propose a simple and effective Inter-Layer Weight Prediction (ILWP) and quantization framework enabling a more compressed neural networks than only applying quantization on the weights of the neural networks. • To further enhance the effectiveness of the proposed ILWP, we devise a new regularization function, denoted as inter-layer loss, that aims to minimize the difference between collocated weight values in the adjacent layers, ing in significant bit saving for non-texture bits (i.e., bits for indices of prediction). • Our comprehensive experiments demonstrate that, the proposed scheme achieves about 53% compression ratio on average in 8-bit quantization at the same accuracy level compared to the traditional quantization method (without prediction) in both MobileNetV1 and MobileNetV2 . Network pruning: Network pruning methods prune the unimportant weight parameters, enabling to reduce the redundancy of weight parameters inherent in neural networks. LeCun et al. (1990b) and reduced the number of weight connections implicitly through setting an proper objective function for training. Han et al. (2015b) successfully removed the unimportant weight connections through certain thresholds for the weight values, showing no harm of accuracy in the state-of-the-art convolutional neural network models. Recently, structured (filter/channel/layerwise) pruning methods have been proposed in and , where a set of weights is pruned based on certain criteria (e.g., the sum of absolute values in the set of weights), demonstrating significantly reduced number of weight parameters and computational costs. use AutoML for channel pruning and their proposed method get accuracy 13.2% more than filter pruning method . Our paper is linked to the pruning methods in perspective of assigning more zero weights for weight compression. Quantization: Quantization reduces the representation bits of original weights in neural networks. proposed a weight quantization using weight discretization in neural networks. Han et al. (2015b) incorporated a vector quantization into the pruning, proving that quantization and pruning can jointly work for weight compression without accuracy degradation. This pruningquantization framework, i.e., called Deep Compression, became a milestone in model compression research of deep neural networks. proposed a fixed-point quantization using a linear scale factor for weight values where bit-widths for quantization are adaptively found for each layer, thus enabling 20% reduction of the weight size in memory without any loss in accuracy compared to the baseline fixed-point quantization method. , and use clipping weights before applying linear quantization and those methods are improve accuracy than linear quantization without clipping. For 1-bit quantization which is named as Binary Neural Networks, several studies (Courbariaux et al. 2015; Courbariaux et al. 2016; Hubara et al. 2017) binarized the weights and/or activations in the process of back-propagation, enjoying considerably reduced usage of memory space and computation overhead. Compared to the aforementioned quantization techniques, our work applies quantization in the combination of the residuals for the weights in inter layers rather than the weight values themselves. Empirically, we found that the residuals tend to produce much larger portion of zero values when quan-tized, since they often follow very narrow Laplace distributions. Therefore, our proposed method can significantly reduce the memory consumption for neural networks as shown in Section 4. Prediction in conventional video coding: Prediction technique is considered one of the most crucial parts in video compression, aiming at minimizing the magnitudes of signals to be encoded by subtracting the input signals to the most similar encoded signals in a set of prediction candidates (Wiegand et al. 2003; Rijkse 1996; Sullivan et al. 2012). The prediction methods can produce the residuals of signals with low magnitudes as well as a large number of non/near zero signals. Therefore, they have effectively been incorporated into transforms and quantization for concentrating powers in low frequency regions and reducing the entropy, respectively. There are two prediction techniques: inter-and intra-predictions. The inter-prediction searches the best prediction signals from the encoded neighbor frames out of the current frame, while the intra-prediction generates a set of prediction signals from the input signals and determine the best prediction (Wiegand et al. 2003; Sullivan et al. 2012). This is because intra frames that use only intra-prediction for compression are used as reference frames for subsequent frames to be predicted. Note that a few studies explored to apply the transform techniques of video and/or image coding to the weight compression problem in neural networks. and applied DCT (Discrete Cosine Transform) used in the JPEG (Joint Picture Encoding Group) algorithm to the model compression problem of deep neural networks such that the energy of weight values became concentrated in low frequency regions, thus producing more non/near-zero DCT coefficients for the weights. Compared to the aforementioned papers, our work does not adopt transform techniques to reduce model sizes, because the transforms introduce much computation in inference, decreasing the effectiveness of the weight compression in practical applications. In this paper, we found out that the inter-prediction technique can play a crucial role for weight compression under the SVWH condition (i.e., the weights between adjacent layers tend to be similar). The proposed inter-layer loss reinforces SVWH for reducing the non-texture bits in the training process. As a , the proposed inter-prediction and quantization framework for weight compression yields impressive compression performance enhancement at the similar accuracy level compared to the baseline models. Figure 1-(a) shows an simple example of the proposed ILWP method with a full search strategy (ILWP-FSS). The full search strategy (FSS) indicates that, for the j-th weight in the kernel of the i-th convolution layer (K i,j), it searches the most similar weight kernel (i.e., K u,v in Figure 1 -(a)) in the range of [1, i − 1]-th layers given a pre-trained neural network model. We then compute the residuals (R i,j) between the current weight and the best prediction, which is finally quantized in a certain bit representation and stored in memory space with the index of the best prediction (i.e., (u, v) in Figure 1-(a) ). The proposed ILWP is performed from the second layer to the last layer in a sequential manner. It should be noted that, because of the large portion of non-texture bits (indices of the best prediction), ILWP-FSS tends to produce more bits (both texture (residual) and non-texture bits) than the standard weight values without ILWP, which makes ILWP worthless in compressing the weights. In the next subsection, we describe how SVWH solves this problem effectively. As shown in Figure 2, the dominant portions of the best predictions in the current (i-th layer) layer are obtained from its previous ((i − 1)-th layer) layers, being consistently observed in all the layers of the neural networks. From these observations, we claim that the adjacent layers tend to consists of the similar weight values, leading us to propose SVWH. The proposed SVWH can be mathematically expressed as where P[·] is a probability, L(·, ·) is a distance between two kernels, and (u, v) are the indices of the layer and kernel, where 1 ≤ u < i − 1 (See Figure 1). Inspired by SVWH, we propose an enhanced version of ILWP, i.e. ILWP with a local search strategy (ILWP-LSS) that finds the best prediction only from the the previous layer (See Figure 1-(b) ). The local search strategy (LSS) can effectively reduce the non-texture bits, because the non-texture bits for the layer index are not required. Moreover, the LSS allows the network to keep the weights only in the previous one layer in inference, thus reducing the memory buffer for keeping the weights. Our experimental in Section 4 show that, the local search strategy enhances the compression performance compared to the FSS in the ILWP framework of deep neural networks. In this paper, to further increase the effectiveness of the proposed ILWP, we devise a new regularization loss, namely inter-layer loss. To further exploit SVWH, we propose a new inter-layer loss which makes the collocated filter weights between the adjacent layers have almost the same values. We find out that our ILWP can more effectively be applied to the depth-wise convolution (3 × 3 spatial convolution) layer in the depth-wise separable convolution block , compared to the traditional 3D convolutions (3 × 3 × C convolution). This is because high dimensionality of the weights in the traditional 3D convolution filter tends to hinder finding out the best prediction sharing strong similarity. This can introduce a longer tail and wider shape of the Laplace distribution for the residuals of the weights, thus decreasing the compression efficiency. Moreover, it is not possible to predict the weight kernels having different channel dimensions from the current weight kernel. This can limit the usability of the proposed ILWP. On the other hand, the depth-wise convolution consists only of nine elements, and all the depthwise convolutions have canonical kernel shapes with 3 × 3 size in whole networks. Also, since the point-wise convolution (1 × 1 convolution) learns channel correlations for the output features of the depth-wise convolution layer, forcing the collocated weights to be similar does hardly affect the accuracy of neural networks. These characteristics of depth-wise convolution enhance the usability of the proposed ILWP in the weight compression of neural networks. Moreover, it becomes more popular to use depth-wise separable convolutions in the recent neural network architectures, due to its high efficiency (Kaiser et al. 2017 ; ; Ma et al. 2018). Therefore, we apply the proposed method into the spatial convolution in the depth-wise separable convolution block. Our proposed inter-layer loss can dramatically eliminate the non-texture bits and is defined as follows: where Z is the number of weights to be predicted, and N is the number of depth-wise convolution layers in a whole neural network. In Eq., v is the index in the previous layer which is equal to j mod c i−1 in the inter layer loss. The proposed loss function in Eq. not only regularizes the weights but also allows us to eliminate all the non-texture bits for indices of the best predictions, since the network can always predict the weight values of the current layer from the collocated weights in its previous layer. For training the neural networks with our proposed loss, our total loss is formulated as follows: where L cls is the conventional classification loss using the cross-entropy loss . λ in Eq. is the control parameter for the inter-layer loss and is set to 1 over all experiments. From our comprehensive experiments, we found that setting λ in Eq. to 1 is suitable to match the trade-off between the performance of neural networks and the parameter size of neural networks (See Figure 4 in Section 4.1). Through our new inter-layer loss, SVWH is explicitly controlled in a more elaborate manner as shown in Figure 3 that shows the Heatmaps for the percentage in the number of best predictions with respect to minimizing L 1 distance between the source layer (x-axis) and the target layer (y-axis) in MobileNet, MobileNetV2, ShuffleNet, and ShuffleNetV2 trained with the proposed inter-layer loss. Finally, the reconstruction of the current weight kernel is performed as follow: whereR i,j is the quantized residual at the i-th layer and j-th filter position. K in Eq. is the final reconstruction of K. Note that, to reconstruct the current weights, the weights only in the previous layer (i.e.,K i−1,v in Eq.) are required (See Figure 1-(c) ). The residuals of the weights are quantized through a linear quantization technique, and then saved using Huffman coding for the purpose of efficient model capacity (b). Thanks to the prediction scheme, our method usually remains more high-valued non-zero weights than the traditional quantization methods. Since high weight values importantly contribute to the prediction performance (b), our method can retain both high accuracy and weight compression performance. The weight kernels in the first layer are not quantized since it is as important as Intra-frames (i.e. reference frames) in a group of pictures of the video coding scheme . If the weight kernels in the first layer are quantized, the weight kernels in subsequent layers which are predicted based on the weight kernels in the first layer are negatively affected, leading to accuracy drop in neural networks. In this section, we describe and prove the superiority of the proposed method by applying it on image classification tasks, specifically for CIFAR-10 and CIFAR-100 datasets. For securing generality of our proposed IWLP, we applied our method in four convolutional neural networks, MobileNet, MobileNetV2, ShuffleNet, and ShuffleNetV2. Four NVidia 2080-Ti GPU with the Intel i9-7900X CPU are used to perform the experiments. For the hyper-parameter setting in the training process, we set the initial learning rate as 0.1, which is multiplied by 0.98 every epochs. We used Stochastic Gradient Descent (SGD) optimizer with Nesterov momentum factor 0.9. All the neural networks are trained for 200 epochs with a batch size of 256. The baseline model is each aforementioned four convolutional neural networks which is quantized by linear quantization on the weights of depthwise convolutional kernel. In all the experiments, the test accuracy and parameters in kilobyte (KB) are marked from the average of 5 runs. First, we found the optimal λ in Eq. that is suitable to match the trade-off between the accuracy performance and the parameter size of neural networks. Figure 4 shows the parameter sizes and top-1 test accuracy in MobileNet trained on the CIFAR-10 dataset for different λ in Eq. after quantization and Huffman coded. We can see that the case of λ = 1 has the smallest parameter size in neural networks, and slightly lower top-1 test accuracy in the CIFAR-10 dataset. Furthermore, not only MobileNet trained on CIFAR-100, but also other models, i.e., MobileNetV2, ShuffleNet, and ShuffleNetV2 trained on CIFAR-10 and CIFAR-100 datasets also has very similar . So, we experimented by setting the λ in Eq. to 1. Figure 5 shows the parameter size of our proposed methods for different quantization bits in {2, 3, 4, ..., 8} on CIFAR-10 and CIFAR-100 datasets. Figure 5 shows that our proposed ILWP-ILL yields higher compression ratio in the weight parameters. On the other hand, ILWP-FSS and ILWP-LSS have larger parameter sizes than the baseline. This is because both ILWP-FSS and ILWP-LSS contain non-texture bits (bits for indices of the best predictions), ing in bit overheads. For further analysis on the texture and non-texture bits in the proposed ILWP, Table 1 shows the performance comparison of our proposed methods in terms of the parameter sizes for the depthwise convolution kernels and top-1 test accuracy. Due to the inter-layer prediction scheme, all the proposed methods show less amount of texture bits compared to the baseline. However, the total amounts of bits of ILWP-FSS and ILWP-LSS are higher than the baseline model, which is due to the presence of the non-texture bits (u and v in Figure 1). However, ILWP-ILL demonstrated much reduced amount of total bits compared to the baseline, which is due to the property that this method does not require saving any indices of reference kernels while maintaining good accuracy under the SVWH condition. Figure 6 shows the of our proposed methods on CIFAR-10 and CIFAR-100 datasets for the parameter size in kilobytes (KB) and test accuracy. As shown in Figure 6, the ILWP-ILL outperforms the other compared ones in terms of the trade-off between the amounts of compressed bits and accuracy. This is because ILWP-ILL significantly saves the weight bits by eliminating the non- texture bits as well as increasing the effectiveness of the quantization process as the residuals tend to follow much narrower Laplace distributions than the original weight values (See Section 5). Compared to ILWP-FSS, ILWP-LSS shows very slight improvement in trade-off between the size of parameters and accuracy as ILWP-LSS does not store the layer index of the best prediction (u in Figure 1). This is due to the fact that the portion of the bits for the layer indices is much smaller than the portion of the bits for the kernel indices. Compared to the baseline models, it is worth noting that both ILWP-FSS and ILWP-LSS have often worse compression efficiency compared to the baseline. This is because they introduce non-texture bits consisting of a large portion in the total bits for weight parameters. Therefore, it can be concluded that our foundation of SVWH and the proposed inter-layer loss allow the network to make use of full advantages in the inter-layer prediction scheme of the conventional video coding frameworks. Figure 7 compares the distributions for the weights and residuals in all the depth-wise convolution kernels in MobileNet trained on CIFAR-100. Figure 7 -(a), -(b), and -(c) visualize the distribution of the weight kernels in baseline model, the distribution of residuals in ILWP-FSS and the distribution of the residuals in ILWP-ILL, respectively. As shown in Figure 7, ILWP-FSS produces a single modal Laplace distribution. This nice property contributes to high compression capacity for storing the weights in neural networks when these residuals are quantized and Huffman coded. However, this method requires saving the additional indices, leading to consuming more amount of bits. As shown in Figure 7 -(c), it can be observed that for ILWP-ILL, the residuals are located in near-zero positions (about 46% of the weights), following a very sharp Laplace distribution. This indicates that ILWP-ILL with quantization allows neural networks to achieve remarkable weight compression performance by generating a large amount of zero coefficients after quantization. Furthermore, in terms of information theory, the Shannon entropy H x of Laplace distribution is derived as follows 1: where, f L (·) is the probability density function of the Laplace distribution and b and µ is scale and location factors of the Laplace distribution, respectively. So, The Shannon entropy is proportional to scale parameter b, which controls width of Laplace distribution, i.e., small b has a narrow Laplace distribution. As shown in Figure 7, it is observed that distribution of residuals in ILWP-ILL has much narrower Laplace distribution than distribution of residuals in ILWP-FSS. Consequently, the information entropy of the distribution of the residuals in ILWP-ILL is lower than the information entropy of the distribution of the residuals in ILWP-FSS. Meanwhile, the entropy coding as Huffman coding is more compressed in small information entropy. As a , the ILWP-ILL method is more compressed than the ILWP-FSS method after quantization and entropy coding. We propose a new inter-layer weight prediction with inter-layer loss for efficient deep neural networks. Motivated by our observation that the weights in the adjacent layers tend to vary smoothly, we successfully build a new weight compression framework combining the inter-layer weight prediction scheme, the inter-layer loss, quantization and Huffman coding under SVWH condition. Intuitively, our prediction scheme significantly decreases the entropy of the weights by making them much narrower Laplace distributions, thus leading remarkable compression ratio of the weight parameters in neural networks. Also, the proposed inter-layer loss effectively eliminates the nontexture bits for the best predictions. To the best of our knowledge, this work is the first to report the phenomenon of the weight similarities between the neighbor layers and to build a prediction-based weight compression scheme in modern deep neural network architectures.

Represent the short summary for finding the corresponding scientific article (long)

We propose a new compression method, Inter-Layer Weight Prediction (ILWP) and quantization method which quantize the predicted residuals between the weights in convolution layers.

We propose accelerating Batch Normalization (BN) through sampling less correlated data for reduction operations  with regular execution pattern, which achieves up to 2x and 20% speedup for BN itself and the overall training, respectively.