Question: Represent the text to find a very short summary (1-2 sentences max)

Question: We propose a generic framework to calibrate accuracy and confidence (score) of a prediction through stochastic inferences in deep neural networks. We first analyze relation between variation of multiple model parameters for a single example inference and variance of the corresponding prediction scores by Bayesian modeling of stochastic regularization. Our empirical observation shows that accuracy and score of a prediction are highly correlated with variance of multiple stochastic inferences given by stochastic depth or dropout. Motivated by these facts, we design a novel variance-weighted confidence-integrated loss function that is composed of two cross-entropy loss terms with respect to ground-truth and uniform distribution, which are balanced by variance of stochastic prediction scores. The proposed loss function enables us to learn deep neural networks that predict confidence calibrated scores using a single inference. Our algorithm presents outstanding confidence calibration performance and improves classification accuracy with two popular stochastic regularization techniques---stochastic depth and dropout---in multiple models and datasets; it alleviates overconfidence issue in deep neural networks significantly by training networks to achieve prediction accuracy proportional to confidence of prediction. Deep neural networks have achieved remarkable performance in various tasks, but have critical limitations in reliability of their predictions. One example is that inference are often overly confident even for unseen or tricky examples; the maximum scores of individual predictions are very high even for out-of-distribution examples and consequently distort interpretation about the predictions. Since many practical applications including autonomous driving, medical diagnosis, and machine inspection require accurate uncertainty estimation as well as high prediction accuracy for each inference, such an overconfidence issue makes deep neural networks inappropriate to be deployed for real-world problems in spite of their impressive accuracy. Regularization is a common technique in training deep neural networks to avoid overfitting problems and improve generalization accuracy BID18; BID6; BID7. However, their objectives are not directly related to generating score distributions aligned with uncertainty of individual predictions. In other words, existing deep neural networks are inherently poor at calibrating prediction accuracy and confidence. Our goal is to learn deep neural networks that are able to estimate accuracy and uncertainty of each prediction at the same time. Hence, we propose a generic framework to calibrate prediction score (confidence) with accuracy in deep neural networks. Our algorithm starts with an observation that variance of prediction scores measured from multiple stochastic inferences is highly correlated with accuracy and confidence of the prediction based on the average score, where we employ stochastic regularization techniques such as stochastic depth or dropout to obtain multiple stochastic inference . We also interpret stochastic regularization as a Bayesian model, which shows relation between stochastic modeling and stochastic inferences of deep neural networks. By exploiting these properties, we design a loss function to enable deep neural network to predict confidence-calibrated scores based only on a single prediction, without stochastic inferences. Our contribution is summarized below:• We provide a generic framework to estimate uncertainty of a prediction based on stochastic inferences in deep neural networks, which is motivated by empirical observation and theoretical analysis.• We design a variance-weighted confidence-integrated loss function in a principled way without hyper-parameters, which enables deep neural networks to produce confidencecalibrated predictions even without stochastic inferences.• The proposed framework presents outstanding performance to reduce overconfidence issue and estimate accurate uncertainty in various architectures and datasets. The rest of the paper is organized as follows. We first discuss prior research related to our algorithm, and describe theoretical for Bayesian interpretation of our approach in Section 2 and 3, respectively. Section 4 presents our confidence calibration algorithm through stochastic inferences, and Section 5 illustrates experimental . Uncertainty estimation is a critical problem in deep neural networks and receives growing attention from machine learning community. Bayesian approach is a common tool to provide a mathematical framework for uncertainty estimation in deep neural networks. However, exact Bayesian inference is not tractable in deep neural networks due to its high computational cost, and various approximate inference techniques-, Laplace approximation and variational inference BID0 BID3 and deep ensembles BID8. All the post-processing methods require a hold-out validation set to adjust prediction scores after training, and the ensemble-based technique employs multiple models to estimate uncertainty. Stochastic regularization is a common technique to improve generalization performance by injecting random noise to deep neural networks. The most notable method is BID18, which randomly drops their hidden units by multiplying Bernoulli random noise. There exist several variants, for example, dropping weights BID21 or skipping layers BID6. Most stochastic regularization methods exploit stochastic inferences during training, but perform deterministic inferences using the whole network during testing. On the contrary, we also use stochastic inferences to obtain diverse and reliable outputs during testing. Although the following works do not address uncertainty estimation, their main idea is relevant to our objective. Label smoothing BID19 encourages models to be less confident, by preventing a network from assigning the full probability to a single class. The same loss function is discussed to train confidence-calibrated classifiers in BID9, but it focuses on how to discriminate in-distribution and out-of-distribution examples, rather than estimating uncertainty or alleviating miscalibration of in-distribution examples. On the other hand, BID15 claims that blind label smoothing and penalizing entropy enhances accuracy by integrating loss functions with the same concept with BID19; BID9, but improvement is marginal in practice. This section describes Bayesian interpretation of stochastic regularization in deep neural networks, and discusses relation between stochastic regularization and uncertainty modeling. Deep neural networks are prone to overfit due to their large number of parameters, and various regularization techniques including weight decay, dropout BID18, and batch normalization BID7 have been employed to alleviate the issue. One popular class of regularization techniques is stochastic regularization, which introduces random noise to a network for perturbing its inputs or weights. We focus on the multiplicative binary noise injection, where random binary noise is applied to the inputs or weights by elementwise multiplication since such stochastic regularization techniques are widely used BID18; BID21; BID6. Note that input perturbation can be reformulated as weight perturbation. For example, dropout-binary noise injection to activations-is intertpretable as weight perturbation that masks out all the weights associated with the dropped inputs. Therefore, if a classification network modeling p(y|x, θ) with parameters θ is trained with stochastic regularization methods by minimizing the cross entropy loss, its objective can be defined by DISPLAYFORM0 whereω i = θ i is a set of perturbed parameters by elementwise multiplication with random noise sample i ∼ p, and (x i, y i) ∈ D is a pair of input and output in training dataset D. Note thatω i is a random sample from p(ω) given by the product of the deterministic parameter θ and a random noise i.At inference time, the network is parameterized by the expectation of the perturbed parameters, DISPLAYFORM1 Given the dataset D with N examples, Bayesian objective is to estimate the posterior distribution of the model parameter, denoted by p(ω|D), to predict a label y for an input x, which is given by DISPLAYFORM0 A common technique for the posterior estimation is variational approximation, which introduces an approximate distribution q θ (ω) and minimizes Kullback-Leibler (KL) divergence with the true posterior D KL (q θ (ω)||p(ω|D)) as follows: DISPLAYFORM1 The intractable integral and summation over the entire dataset in Equation 4 is approximated by Monte Carlo method and mini-batch optimization ing in DISPLAYFORM2 DISPLAYFORM3 is a sample from the approximate distribution, S is the number of samples, and M is the size of a mini-batch. Note that the first term is data likelihood and the second term is divergence of the approximate distribution with respect to the prior distribution. Suppose that we train a classifier with 2 regularization by a stochastic gradient descent method. Then, the loss function in Equation 1 is rewritten aŝ DISPLAYFORM0 where 2 regularization is applied to the deterministic parameters θ with weight λ. Optimizing this loss function is equivalent to optimizing Equation 5 if there exists a proper prior p(ω) and q θ (ω) is approximated as a Gaussian mixture distribution BID1. Note that Gal & DISPLAYFORM1 Following BID1 and BID20, we estimate the predictive mean and uncertainty by Monte Carlo approximation by drawing binary noise samples DISPLAYFORM2 as DISPLAYFORM3 where y = (y 1, . . ., y C) denotes a vector of C class labels. Note that the binary noise samples realize stochastic inferences such as stochastic depth and dropout by elementwise multiplication with model parameter θ. Equation 8 means that the average prediction and its variance can be computed directly from multiple stochastic inferences. We present a novel confidence calibration technique for prediction in deep neural networks, which is given by a variance-weighted confidence-integrated loss function. We present our observation that variance of multiple stochastic inferences is closely related to accuracy and confidence of predictions, and provide an end-to-end training framework for confidence self-calibration. Then, prediction accuracy and uncertainty are directly accessible from the predicted scores obtained from a single forward pass. This section presents our observation from stochastic inferences and technical details about our confidence calibration technique. Equation FORMULA9 suggests that variation of models 1 is correlated to variance of multiple stochastic predictions for a single example. In other words, by observing variation of multiple stochastic inferences, we can estimate accuracy and uncertainty of the prediction given by average of the stochastic inferences corresponding to an example. FIG0 presents how variance of multiple stochastic inferences given by stochastic depth or dropout is related to accuracy and confidence of the corresponding average prediction, where the confidence is measured by the maximum score of the average prediction. In the figure, accuracy and score of each bin are computed with the examples belonging to the corresponding bin of the normalized variance. We present from CIFAR-100 with ResNet-34 and VGGNet with 16 layers. The histograms illustrate the strong correlation between the predicted variance and the reliability-accuracy and confidence-of a prediction, and between accuracy and prediction. These suggest that one can disregard examples based on their prediction variances. Note that variance computation with more stochastic inferences provides more reliable estimation of accuracy and confidence. We first design a simple loss function for accuracy-score calibration by augmenting a confidenceintegrated loss L U to the standard cross-entropy loss term, which is given by DISPLAYFORM0 where H is the cross entropy loss function, p GT is the ground-truth distribution, p(y|x i, θ) is the predicted distribution with model parameter θ, U(y) is the uniform distribution, and ξ is a constant. The loss denoted by L 1 (·) is determined based on cross-entropy with the ground-truths and KLdivergence from the uniform distribution. The main idea of this loss function is to regularize with the uniform distribution by expecting the score distributions of uncertain examples to be flattened first while the distributions of confident ones remain intact, where the impact of the confidenceintegrated loss term is controlled by a global hyper-parameter β. The proposed loss function is also employed in BID15 to regularize deep neural networks and improve classification accuracy. However, BID15 does not discuss confidence calibration issues. On the other hand, BID9 discusses the same loss function but focuses on differentiating between in-distribution and out-of-distribution examples by measuring loss of each example based only on one of the two loss terms depending on its origin. Contrary to these approaches, we employ the loss function in Equation 9 for estimating prediction confidence in deep neural networks. Although the proposed loss makes sense intuitively, blind selection of a constant β limits its generality. Hence, we propose a more sophisticated confidence loss term by leveraging variance of multiple stochastic inferences. The strong correlation of accuracy and confidence with predicted variance observed in FIG0 shows great potential to make confidence-calibrated prediction by stochastic inferences. However, variance computation involves multiple stochastic inferences by executing multiple forward passes. Note that this property incurs additional computational cost and may produce inconsistent . To overcome these limitations, we propose a generic framework for training accuracy-score calibration networks whose prediction score from a single forward pass directly provides confidence of the prediction. In this framework, we combine two complementary loss terms as in Equation 9, but they are balanced by the variance measured by multiple stochastic inferences. Our variance-weighted confidence-integrated loss L(·) for the whole training data (x i, y i) ∈ D is defined by a linear interpolation of the standard cross-entropy loss with ground-truth L GT (·) and the cross-entropy with the uniform distribution L U (·), which is formally given by DISPLAYFORM0 where α i ∈ is a normalized variance,ω i,j (= θ i,j) is a sampled model parameter with binary noise for stochastic prediction, T is the number of stochastic inferences, and ξ i is a constant. The two terms in our variance-weighted confidence-integrated loss pushes the network toward opposite directions; the first term encourages the network to produce a high score for the ground truth label while the second term forces the network to predict the uniform distribution. These terms are linearly interpolated by a balancing coefficient α i, which is the normalized variance of individual example obtained by multiple stochastic inferences. Note that the normalized variance α i is unique for each training example and is used to measure model uncertainty. Therefore, optimizing our loss function produces gradient signals, forcing the prediction toward the uniform distribution for the examples with high uncertainty derived by high variance while intensifying prediction confidence of the examples with low variance. After training models in our framework, prediction of each testing example is made by a single forward pass. Unlike the ordinary models, however, a prediction score of our model is well-calibrated and represents confidence of the prediction, which means that we can rely more on the predictions with high scores. There are several score calibration techniques BID3; BID23; Naeini et al. FORMULA0; BID14 by adjusting confidence scores through postprocessing, among which BID3 proposes a method to calibrate confidence of predictions by scaling logits of a network using a global temperature τ. The scaling is performed before applying the softmax function, and τ is trained with validation dataset. As discussed in BID3, this simple technique is equivalent to maximize entropy of the output distribution p(y i |x i). It is also identical to minimize KL-divergence D KL (p(y i |x i)||U(y)) because DISPLAYFORM0 where ξ c is a constant. We can formulate another confidence-integrated loss with the entropy as DISPLAYFORM1 where γ is a constant. Equation 12 suggests that temperature scaling in BID3 is closely related to our framework. We choose two most widely adapted deep neural network architectures: ResNet and VGGNet. VGG architecture follows BID17, where we employ dropout BID18 before every fc layer except for the classification layer. In ResNet, instead of stacking conv layers directly, outputs of residual blocks are added to the input feature representation by residual connections as proposed in BID4. Stochastic depth BID6 is used for stochastic regularization in ResNet. Note that, as discussed in Section 3.3, both dropout and stochastic depth inject multiplicative binary noise to within-layer activations or residual blocks, they are equivalent to noise injection into network weights. Hence, training with 2 regularization term enables us to interpret stochastic depth and dropout by Bayesian models. We evaluate the proposed framework on two benchmarks, Tiny ImageNet and CIFAR-100. Tiny ImageNet contains 64 × 64 images with 200 object labels whereas CIFAR-100 has 32 × 32 images of 100 objects. There are 500 training images per class in both datasets. For testing, we use the validation set of Tiny ImageNet and the test set of CIFAR-100, which contain 50 and 100 images per class, respectively. To use the same network for two benchmarks, we resize images in Tiny ImageNet into 32 × 32.All networks are trained with stochastic gradient decent with the momentum of 0.9 for 300 epochs. We set the initial learning rate to 0.1 and exponentially decay it with factor of 0.2 at epoch 60, 120, 160, 200 and 250. Each batch consists of 64 and 256 training examples for ResNet and VGG architectures, respectively. To train networks with the proposed variance-weighted confidence-integrated loss, we draw T samples for each input image by default, and compute the normalized variance α by running T forward passes. The number of samples T is set to 5. The normalized variance is estimated based on the variance of Bhattacharyya coefficients between individual predictions and the average prediction. The trained models with the variance-weighted confidence-integrated (VWCI) loss are compared to the models with confidence-integrated (CI) losses for several different constant β's. We measure classification accuracy and expected calibration error (ECE) of the trained models. While classification accuracy shows regularization effect of the confidence-integrated loss term, ECE summarizes miscalibration of a model by measuring discrepancy between confidence and accuracy. Specifically, let B m be a set of indices of test examples whose scores for the ground-truth labels fall into the score interval (DISPLAYFORM0, where M is the number of bins. Then, ECE is formally defined by DISPLAYFORM1 where N is the number of the test samples. Also, accuracy and confidence of each bin are given by DISPLAYFORM2 whereŷ i and y i are predicted and true label of the i-th example and p i is its predicted confidence. TAB1 presents of ResNet-34 and VGG-16 on both datasets. We observe that baseline methods with stochastic inferences reduce calibration error and the reduction becomes more significant in proportion to number of inferences. These imply benefit of stochastic inference for confidence calibration, and reflect performance of methods by multiplicative noise in BID1 ; BID11 . The models trained with VWCI loss consistently outperform baselines and is competitive to models with CI loss on both classification accuracy and confidence calibration. Stochastic inference and variance-driven weight allow us to measure uncertainty for each instance, and enable two oppositional terms to be well balanced by the measured uncertainty. The confidence loss term regularizes the network by forcing predictions to uniform distribution, and the proper estimation of its coefficient leads to accuracy gain and confidence calibration. Note that, by assigning a low coefficient to the loss term for a confident example, the network allows the example to remain confident whereas a high weight for an uncertain example reduces confidence of the prediction. The CI loss has a confidence loss term with fixed coefficient β. The networks trained with proper β show impressive improvement on both criteria, but their performance is sensitive to choice of β as this strategy ignores predictive uncertainty for confidence loss; an inappropriate choice of β even worsens accuracy and calibration error, e.g., ResNet-34 trained with CI[β = 0.01] on Tiny ImageNet. Also, there seems to be no single β that is globally optimal across architectures and benchmark datasets. For instance, training the network with CI[β = 0.01] on Tiny ImageNet gives the worst accuracy with ResNet-34 and the best accuracy with VGG-16. In the experiments, the CI loss often works well on CIFAR-100 due to high accuracy. The majority of examples are classified correctly and the overconfident property of deep neural networks do little harm for confidence calibration. Specifically, CI loss sometimes achieves slightly better performance than VWCI with a certain fixed coefficient β because the measured normalized variance by stochastic inferences and its range are small. On the other hand, in Tiny ImageNet dataset, performance of VMCI is consistently better than CI because Tiny ImageNet is substantially more challenging than CIFAR-100.A critical benefit of our variance-driven weight in the VWCI loss is the capability to maintain examples with high accuracy and high confidence. This is an important property for building real-world decision making systems with confidence interval, where the decisions should be both highly accurate and confident. FIG1 illustrates coverage of test examples varying the confidence threshold, and VWCI shows better coverage than CI because CI pushes all instances to uniform with the same strength β regardless of their uncertainty unlike VWCI. It is also notable that β for the best coverage is different from that for the best accuracy and ECE whereas VWCI balances these based on the predictive uncertainty. These suggest that using the predictive uncertainty for balancing the terms is preferable over setting a constant coefficient in our loss function. More experimental are presented in the supplementary document. We presented a generic framework for uncertainty estimation of a prediction in deep neural networks by calibrating accuracy and score based on stochastic inferences. Based on Bayesian interpretation of stochastic regularization and our empirical observation , we claim that variation of multiple stochastic inferences for a single example is a crucial factor to estimate uncertainty of the average prediction. Motivated by this fact, we design the variance-weighted confidence-integrated loss to learn confidence-calibrated networks and enable uncertainty to be estimated by a single prediction. The proposed algorithm is also useful to understand existing confidence calibration methods in a unified way, and we compared our algorithm with other variations within our framework to analyze their characteristics.

Represent the short summary for finding the corresponding scientific article (long)

We propose a framework to learn confidence-calibrated networks by designing a novel loss function that incorporates predictive uncertainty estimated through stochastic inferences.

Represent the TLDR for finding the corresponding scientific article

This paper 1) characterizes functions representable by ReLU DNNs, 2) formally studies the benefit of depth in such architectures,  3) gives an algorithm to implement empirical risk minimization to global optimality for two layer ReLU nets.


Question: Represent the article for finding a single-sentence TLDR!

Question: Real-life control tasks involve matters of various substances---rigid or soft bodies, liquid, gas---each with distinct physical behaviors. This poses challenges to traditional rigid-body physics engines. Particle-based simulators have been developed to model the dynamics of these complex scenes; however, relying on approximation techniques, their simulation often deviates from real-world physics, especially in the long term. In this paper, we propose to learn a particle-based simulator for complex control tasks. Combining learning with particle-based systems brings in two major benefits: first, the learned simulator, just like other particle-based systems, acts widely on objects of different materials; second, the particle-based representation poses strong inductive bias for learning: particles of the same type have the same dynamics within. This enables the model to quickly adapt to new environments of unknown dynamics within a few observations. We demonstrate robots achieving complex manipulation tasks using the learned simulator, such as manipulating fluids and deformable foam, with experiments both in simulation and in the real world. Our study helps lay the foundation for robot learning of dynamic scenes with particle-based representations. Objects have distinct dynamics. Under the same push, a rigid box will slide, modeling clay will deform, and a cup full of water will fall with water spilling out. The diverse behavior of different objects poses challenges to traditional rigid-body simulators used in robotics BID31 BID30. Particle-based simulators aim to model the dynamics of these complex scenes BID18; however, relying on approximation techniques for the sake of perceptual realism, their simulation often deviates from real world physics, especially in the long term. Developing generalizable and accurate forward dynamics models is of critical importance for robot manipulation of distinct real-life objects. We propose to learn a differentiable, particle-based simulator for complex control tasks, drawing inspiration from recent development in differentiable physical engines BID0 BID3. In robotics, the use of differentiable simulators, together with continuous and symbolic optimization algorithms, has enabled planning for increasingly complex whole body motions with multi-contact and multi-object interactions BID32 ). Yet these approaches have only tackled local interactions of rigid bodies. We develop dynamic particle interaction networks (DPINets) for learning particle dynamics, focusing on capturing the dynamic, hierarchical, and long-range interactions of particles FIG0 -c). DPI-Nets can then be combined with classic perception and gradient-based control algorithms for robot manipulation of deformable objects FIG0 ).Learning a particle-based simulator brings in two major benefits. First, the learned simulator, just like other particle-based systems, acts widely on objects of different states. DPI-Nets have successfully captured the complex behaviors of deformable objects, fluids, and rigid-bodies. With learned DPINets, our robots have achieved success in manipulation tasks that involve deformable objects of complex physical properties, such as molding plasticine to a target shape. Our project page: http://dpi.csail.mit.edu Perception and control with the learned model. Our system first reconstructs the particle-based shape from visual observation. It then uses gradient-based trajectory optimization to search for the actions that produce the most desired output. Second, the particle-based representation poses strong inductive bias for learning: particles of the same type have the same dynamics within. This enables the model to quickly adapt to new environments of unknown dynamics within a few observations. Experiments suggest that DPI-Nets quickly learn to adapt to characterize a novel object of unknown physical parameters by doing online system identification. The adapted model also helps the robot to successfully manipulate object in the real world. DPI-Nets combine three key features for effective particle-based simulation and control: multi-step spatial propagation, hierarchical particle structure, and dynamic interaction graphs. In particular, it employs dynamic interaction graphs, built on the fly throughout manipulation, to capture the meaningful interactions among particles of deformable objects and fluids. The use of dynamic graphs allows neural models to focus on learning meaningful interactions among particles, and is crucial for obtaining good simulation accuracy and high success rates in manipulation. As objects deform when robots interact with them, a fixed interaction graph over particles is insufficient for robot manipulating non-rigid objects. Experiments demonstrate that DPI-Nets significantly outperform interaction networks BID0, HRN BID19, and a few other baselines. More importantly, unlike previous paper that focused purely on forward simulation, we have applied our model to downstream control tasks. Our DPI-Nets enable complex manipulation tasks for deformable objects and fluids, and adapts to scenarios with unknown physical parameters that need to be identified online. We have also performed real-world experiments to demonstrate our model's generalization ability. Differentiable physics simulators. Researchers have developed many differentiable physical simulators BID7 BID6 BID31, among which some can also provide analytical gradients BID30. In particular, BID0 and BID3 have explored learning a simulator from data by approximating object interactions with neural networks. BID16 proposed learning to propagate signals along the interaction graph and extended to partially observable scenarios. These methods mostly focus on modeling rigid body dynamics. Differentiable simulators for deformable objects have been less studied. Recently, BID25 proposed SPNets for differentiable simulation of position-based fluids BID17 ). An inspiring concurrent work from BID19 explored learning to approximate particle dynamics of deformable shapes with the Hierarchical Relation Network (HRN). Compared with these papers, we introduce state-specific modeling and dynamic graphs for accurate forward prediction for different states of matter (rigid bodies, deformable shapes, fluids). We also demonstrate how the learned dynamics model can be used for control in both simulation and real world. Our approach is also complementary to some recent work on learning to discover the interaction graphs BID33 BID13. Our model can also be naturally augmented with a perception module to handle raw visual input, as suggested by BID34; BID35 BID9.Model-predictive control with a differentiable simulator. Many recent papers have studied model-predictive control with deep networks BID15 BID10 BID20 BID8 BID28. They often learn an abstract state transition function, instead of an explicit account of the environment BID21, and then use the learned function to facilitate training of a policy network. A few recent papers have employed analytical, differentiable simulators (de Avila BID5 BID25 for control problems, such as tool manipulation and tool-use planning BID32 . Our model builds on and extends these approaches by learning a general physics simulator that takes raw object observations (e.g., positions, velocities) of each particle as input. We then integrate it into classic trajectory optimization algorithms for control. Compared with pure analytical simulators, our learned simulator can better generalize to novel testing scenarios where object and environment parameters are unknown. A few papers have explored using interaction networks for planning and control. They often learn a policy based on interaction networks' rollouts BID11. In contrast, our model learns a dynamics simulator and directly optimizes trajectories for continuous control. Recently, BID24 have applied interaction networks for control, and BID16 have further extended interaction nets to handle instance signal propagation for controlling multiple rigid bodies under partial observations. Compared with them, our dynamic particle interaction network simulate and control deformable, particle-based objects, using dynamic graphs to tackle scenes with complex object interactions. We first describe how interaction networks BID0 represent the physical system; we then extend them for particle-based dynamics. The interactions within a physical system are represented as a directed graph, G = O, R, where vertices O = {o i} represent objects and edges R = {r k} represent relations. Specifically, DISPLAYFORM0, where x i = q i,q i is the state of object i, containing its position q i and velocityq i. a o i denotes its attributes (e.g., mass, radius). For relation, we have DISPLAYFORM1 where u k is the receiver, v k is the sender, and both are integers. a r k is the type and attributes of relation k (e.g., collision, spring connection).The goal is to build a learnable physical engine to capture the underlying physical interactions using function approximators φ. The learned model can then be used to infer the system dynamics and predict the future from the current interaction graph as G t+1 = φ(G t), where G t denotes the scene state at time t. Interaction networks. BID0 proposed interaction networks (IN), a generalpurpose, learnable physics engine that performs object-and relation-centric reasoning about physics. INs define an object function f O and a relation function f R to model objects and their relations in a compositional way. The future state at time t + 1 is predicted as DISPLAYFORM2 denotes object i at time t, u k and v k are the receiver and sender of relation r k respectively, and N i denotes the relations where object i is the receiver. Propagation networks. A limitation of INs is that at every time step t, it only considers local information in the graph G and cannot handle instantaneous propagation of forces, which however is a common phenomenon in rigid-body dynamics. BID16 proposed propagation networks to handle the instantaneous propagation of forces by doing multi-step message passing. Specifically, they first employed the ideas on fast training of RNNs BID14 BID1 to encode the shared information beforehand and reuse them along the propagation steps. The encoders for objects are denoted as f DISPLAYFORM3 At time t, denote the propagating influence from relation k at propagation step l as e l k,t, and the propagating influence from object i as h l i,t. For step 1 ≤ l ≤ L, propagation can be described asStep 0: h DISPLAYFORM4 Step DISPLAYFORM5 DISPLAYFORM6 Output: DISPLAYFORM7 where f O denotes the object propagatorand f R denotes the relation propagator. Particle-based system is widely used in physical simulation due to its flexibility in modeling various types of objects BID18. We extend existing systems that model object-level interactions to allow particle-level deformation. Consider object set {o i}, where each object DISPLAYFORM0.. |oi| is represented as a set of particles. We now define the graph on the particles and the rules for influence propagation. Dynamic graph building. The vertices of the graph are the union of particles for all objects DISPLAYFORM1..|O|,k=1...|oi|. The edges R between these vertices are dynamically generated over time to ensure efficiency and effectiveness. The construction of the relations is specific to environment and task, which we'll elaborate in Section 4. A common choice is to consider the neighbors within a predefined distance. An alternative is to build a static, complete interaction graph, but it has two major drawbacks. First, it is not efficient. In many common physical systems, each particle is only interacting with a limited set of other particles (e.g., those within its neighborhood). Second, a static interaction graph implies a universal, continuous neural function approximator; however, many physical interactions involve discontinuous functions (e.g. contact). In contrast, using dynamic graphs empowers the model to tackle such discontinuity. Hierarchical modeling for long-range dependence. Propagation networks BID16 require a large L to handle long-range dependence, which is both inefficient and hard to train. Hence, we add one level of hierarchy to efficiently propagate the long-range influence among particles BID19. For each object that requires modeling of the long-range dependence (e.g. rigid-body), we cluster the particles into several non-overlapping clusters. For each cluster, we add a new particle as the cluster's root. Specifically, for each object o i that requires hierarchical modeling, the corresponding roots are denoted asõ i = {õ k i} k=1...|õi|, and the particle set containing all the roots is denoted asÕ = {õ k i} i=1...|O|,k=1...|õi|. We then construct an edge set R LeafToRoot that contains directed edges from each particle to its root, and an edge set R RootToLeaf containing directed edges from each root to its leaf particles. For each object that need hierarchical modeling, we add pairwise directed edges between all its roots, and denote this edge set as R RootToRoot.We employ a multi-stage propagation paradigm: first, propagation among leaf nodes, φ LeafToLeaf (O, R); second, propagation from leaf nodes to root nodes, φ LeafToRoot (O ∪ O, R LeafToRoot); third, propagation between roots, φ RootToRoot (Õ, R RootToRoot); fourth, propagation from root to leaf, φ RootToLeaf (O ∪Õ, R RootToLeaf). The signals on the leaves are used to do the final prediction. Applying to objects of various materials. We define the interaction graph and the propagation rules on particles for different types of objects as follows:• Rigid bodies. All the particles in a rigid body are globally coupled; hence for each rigid object, we define a hierarchical model to propagate the effects. After the multi-stage propagation, we average the signals on the particles to predict a rigid transformation (rotation and translation) for the object. The motion of each particle is calculated accordingly. For each particle, we also include its offset to the center-of-mass to help determine the torque.• Elastic/Plastic objects. For elastically deforming particles, only using the current position and velocity as the state is not sufficient, as it is not clear where the particle will be restored after the deformation. Hence, we include the particle state with the resting position to indicate the place where the particle should be restored. When coupled with plastic deformation, the resting position might change during an interaction. Thus, we also infer the motion of the resting position as a part of the state prediction. We use hierarchical modeling for this category but predict next state for each particles individually.• Fluids. For fluid simulation, one has to enforce density and incompressibility, which can be effectively achieved by only considering a small neighborhood for each particle BID17. Therefore, we do not need hierarchical modeling for fluids. We build edges dynamically, connecting a fluid particle to its neighboring particles. The strong inductive bias leveraged in the fluid particles allows good performance even when tested on data outside training distributions. For the interaction between different materials, two directed edges are generated for any pairs of particles that are closer than a certain distance. Model-based methods offer many advantages when comparing with their model-free counterparts, such as generalization and sample efficiency. However, for cases where an accurate model is hard to specify or computationally prohibitive, a data-driven approach that learns to approximate the underlying dynamics becomes useful. Function approximators such as neural networks are naturally differentiable. We can rollout using the learned dynamics and optimize the control inputs by minimizing a loss between the simulated and a target configuration. In cases where certain physical parameters are unknown, we can perform online system identification by minimizing the difference between the model's prediction and the reality. An outline of our algorithm can be found in Section A.Model predictive control using shooting methods. Let's denote G g as the goal andû 1:T be the control inputs, where T is the time horizon. The control inputs are part of the interaction graph, such as the velocities or the initial positions of a particular set of particles. We denote the ing trajectory after applyingû as G = {G i} i=1:T. The task here is to determine the control inputs as to minimize the distance between the actual outcome and the specified goal L goal (G, G g).Our dynamic particle interaction network does forward simulation by taking the dynamics graph at time t as input, and produces the graph at next time step,Ĝ t+1 = Φ(G t), where Φ is implemented as DPI-Nets as described in the previous section. Let's denote the the history until time t as G = {G i} i=1...t, and the forward simulation from time step t asĜ = {Ĝ i} i=t+1...T. The loss L goal (Ḡ ∪Ĝ, G g) can be used to update the control inputs by doing stochastic gradient descent (SGD). This is known as the shooting method in trajectory optimization BID29 ).The learned model might deviate from the reality due to accumulated prediction errors. We use Model-Predictive Control (MPC) BID2 to stabilize the trajectory by doing forward simulation and updating the control inputs at every time step to compensate the simulation error. Online adaptation. In many real-world cases, without actually interacting with the environment, inherent attributes such as mass, stiffness, and viscosity are not directly observable. DPI-Nets can estimate these attributes on the fly with SGD updates by minimizing the distance between the predicted future states and the actual future states L state (Ĝ t, G t). We evaluate our method on four different environments containing different types of objects and interactions. We will first describe the environments and show simulation . We then present how the learned dynamics helps to complete control tasks in both simulation and the real world. FluidFall (Figure 2a). Two drops of fluids are falling down, colliding, and merging. We vary the initial position and viscosity for training and evaluation. BoxBath (Figure 2b). A block of fluids are flushing a rigid cube. In this environment, we have to model two different materials and the interactions between them. We randomize the initial position of the fluids and the cube to test the model's generalization ability. FluidShake (Figure 2c). We have a box of fluids and the box is moving horizontally, The speed of the box is randomly selected at each time step. We vary the size of the box and volume of the fluids to test generalization. RiceGrip (Figure 2d). We manipulate an object with both elastic and plastic deformation (e.g., sticky rice). We use two cuboids to mimic the fingers of a parallel gripper, where the gripper is initialized at a random position and orientation. During the simulation of one grip, the fingers will move closer to each other and then restore to its original positions. The model has to learn the interactions between the gripper and the "sticky rice", as well as the interactions within the "rice" itself. We use all four environments in evaluating our model's performance in simulation. We use the rollout MSE as our metric. We further use the latter two for control, because they involve fully actuated external shapes that can be used for object manipulation. In FluidShake, the control task requires determining the speed of the box at each time step, in order to make the fluid match a target configuration within a time window; in RiceGrip, the control task corresponds to select a sequence of grip configurations (position, orientation, closing distance) to manipulate the deformable object as to match a target shape. The metric for performance in control is the Chamfer distance between the manipulation and the target configuration. We present implementation details for dynamics learning in the four environment and perform ablation studies to evaluate the effectiveness of the introduced techniques. Implementation details. For FluidFall, we dynamically build the interaction graph by connecting each particle to its neighbors within a certain distance d. No hierarchical modeling is used. For BoxBath, we model the rigid cube as in Section 3.2, using multi-stage hierarchical propagation. Two directed edges will be constructed between two fluid particles if the distance between them is smaller than d. Similarly, we also add two directed edge between rigid particles and fluid particles when their distance is smaller than d. For FluidShake, we model fluid as in Section 3.2. We also add five external particles to represent the walls of the box. We add a directed edge from the wall particle to the fluid particle when they are closer than d. The model is a single propagation network, where the edges are dynamically constructed over time. For RiceGrip, we build a hierarchical model for rice and use four propagation networks for multi-stage effect propagation (Section 3.2). The edges between the "rice" particles are dynamically generated if two particles are closer than d. Similar to FluidShake, we add two external particles to represent the two "fingers" and add an edge from the "finger" to the "rice" particle if they are closer than the Figure 2: Qualitative on forward simulation. We compare the ground truth (GT) and the rollouts from HRN BID19 and our model (DPI-Net) in four environments (FluidFall, BoxBath, FluidShake, and RiceGrip). The simulations from our DPI-Net are significantly better. We provide zoom-in views for a few frames to show details. Please see our video for more empirical .distance d. As "rice" can deform both elastically and plastically, we maintain a resting position that helps the model restore a deformed particle. The output for each particle is a 6-dim vector for the velocity of the current observed position and the resting position. More training details for each environment can be found in Section D. Details for data generation are in Section C. Table 1: Quantitative on forward simulation. MSE (×10 −2) between the ground truth and model rollouts. The hyperparameters used in our model are fixed for all four environments. FluidFall and FluidShake involve no hierarchy, so DPI-Net performs the same as the variant without hierarchy. DPI-Net significantly outperforms HRN BID19 in modeling fluids (BoxBath and FluidShake) due to the use of dynamic graphs. Results. Qualitative and quantitative are in Figure 2 and Table 1. We compare our method (DPI-Net) with three baselines, Interaction Networks BID0, HRN BID19, and DPI-Net without hierarchy. Note that we use the same set of hyperparameters in our model for all four testing environments. Specifically, Interaction Networks (IN) consider a complete graph of the particle system. Thus, it can only operate on small environments such as FluidFall; it runs out of memory (12GB) for the other three environments. IN does not perform well, because its use of a complete graph makes training difficult and inefficient, and because it ignores influence propagation and long-range dependence. Without a dynamic graph, modeling fluids becomes hard, because the neighbors of a fluid particle changes constantly. Table 1 shows that for environments that involve fluids (BoxBath and FluidShake), DPI-Net performs better than those with a static interaction graph. Our model also performs better in scenarios that involve objects of multiple states (BoxBath, Figure 2b), because it uses state-specific modeling. Models such as HRN BID19 aim to learn a universal dynamics model for all states of matter. It is therefore solving a harder problem and, for this particular scenario, expected to perform not as well. When augmented with state-specific modeling, HRN's performance is likely to improve, too. Without hierarchy, it is hard to capture long-range dependence, leading to performance drop in environments that involve hierarchical object modeling (BoxBath and RiceGrip).Appendix B includes on scenarios outside the training distribution (e.g., more particles). DPI-Net performs well on these out-of-sample cases, successfully leveraging the inductive bias. Ablation studies. We also test our model's sensitivity to hyperparameters. We consider three of them: the number of roots for building hierarchy, the number of propagation steps L, and the size of the neighborhood d. We test them in RiceGrip. As can be seen from the shown in FIG4, DPI-Nets can better capture the motion of the "rice" by using fewer roots, on which the information might be easier to propagate. Longer propagation steps do not necessarily lead to better performance, as they increases training difficulty. Using larger neighborhood achieves better , but makes computation slower. Using one TITAN Xp, each forward step in RiceGrip takes 30ms for d = 0.04, 33ms for d = 0.08, and 40ms for d = 0.12. We also perform experiments to justify our use of different motion predictors for objects of different states. FIG4 shows the of our model vs. a unified dynamics predictor for all objects in BoxBath. As there are only a few states of interest (solids, liquids, and soft bodies), and their physical behaviors are drastically different, it is not surprising that DPI-Nets, with state-specific motion predictors, perform better, and are equally efficient as the unified model (time difference smaller than 3ms per forward step). We leverage dynamic particle interaction network for control tasks in both simulation and real world. Because trajectory optimization using shooting method can easily stuck to a local minimum, we first randomly sample N sample control sequences, and select the best performing one according to the rollouts of our learned model. We then optimize it via shooting method using our model's gradients. We also use online system identification to further improve the model's performance. FIG5 and FIG6 show qualitative and quantitative , respectively. More details of the control algorithm can be found in Section E.FluidShake. We aim to control the speed of the box to match the fluid particles to a target configuration. We compare our method (RS+TO) with random search over the learned model (without trajectory optimization -RS) and Model-free Deep Reinforcement Learning (Actor-Critic method optimized with PPO (RL). FIG6 suggests that our model-based control algorithm outperforms both baselines with a large margin. Also RL is not sample-efficient, requiring more than 10 million time steps to converge while ours requires 600K time steps. RiceGrip. We aim to select a sequence of gripping configurations (position, orientation, and closing distance) to mold the "sticky rice" to a target shape. We also consider cases where the stiffness of the rice is unknown and need to be identified. FIG6 shows that our dynamic particle interaction network with system identification performs the best, and is much more efficient than RL (150K vs. RiceGrip in the real world. We generalize the learned model and control algorithm for RiceGrip to the real world. We first reconstruct object geometry using a depth camera mounted on our Kuka robot using TSDF volumetric fusion BID4 . We then randomly sampled N fill particles within the object mesh as the initial configuration for manipulation. FIG5 and FIG6 shows that, using DPI-Nets, the robot successfully adapts to the real world environment of unknown physical parameters and manipulates a deformable foam into various target shapes. The learned policy in RiceGrip does not generalize to the real world due to domain discrepancy, and outputs invalid gripping configurations. We have demonstrated that a learned particle dynamics model can approximate the interaction of diverse objects, and can help to solve complex manipulation tasks of deformable objects. Our system requires standard open-source robotics and deep learning toolkits, and can be potentially deployed in household and manufacturing environment. Robot learning of dynamic scenes with particle-based representations shows profound potentials due to the generalizability and expressiveness of the representation. Our study helps lay the foundation for it. A CONTROL ALGORITHM Update A by descending with the gradients ∇ A L state (Ĝ t, G t) Forward simulation using the current graphĜ t+1 ← Φ(G t) Make a buffer for storing the simulation G ←Ḡ ∪Ĝ t+1 for i = t + 1,..., T − 1 do Forward simulation:Ĝ j+1 ← Φ(Ĝ j); G ← G ∪Ĝ j+1 end for Updateû t:T by descending with the gradients ∇û t: DISPLAYFORM0 We show our model's performance on fluids, rigid bodies, and deformable objects with a larger number of particles than they have in the training set. Figure 6 shows qualitative and quantitative . Our model scales up well to larger objects. Figure 6: Extrapolate generalization on fluids, rigid bodies, and deformable objects. The performance is evaluated by the MSE (×10 −2) between the ground truth and rollouts from DPI-Nets. The blue bars denote the range of particle numbers that have been seen during training, which indicate interpolation performance. The red bars indicate extrapolation performance that our model can generalize to cases containing two times more particles than cases it has been trained on.clusterPlasticCreep is uniformly sampled between 0.1 and 0.3. The position of the gripper is randomly sampled within a circle of radius 0.5. The orientation of the gripper is always perpendicular to the line connecting the origin to the center of the gripper and the close distance is uniformly sampled between 0.7 to 1.0.Of all the generated data, 90% of the rollouts are used for training, and the rest 10% are used for validation. The models are implemented in PyTorch, and are trained using Adam optimizer BID12 ) with a learning rate of 0.0001. The number of particles and relations might be different at each time step, hence we use a batch size of 1, and we update the weights of the networks once every 2 forward rounds. The neighborhood d is set as 0.08, and the propagation step L is set as 2 for all four environments. For hierarchical modeling, it does not make sense to propagate more than one time between leaves and roots as they are disjoint particle sets, and each propagation stage between them only involves one-way edges; hence φ LeafToLeaf uses L = 2. φ LeafToRoot uses L = 1. φ RootToRoot uses L = 2, and φ RootToLeaf uses L = 1.For all propagation networks used below, the object encoder f FluidFall. The model is trained for 13 epochs. The output of the model is the 3 dimensional velocity, which is multiplied by ∆t and added to the current position to do rollouts. BoxBath. In this environment, four propagation networks are used due to the hierarchical modeling and the number of roots for the rigid cube is set as 8. We have two separate motion predictor for fluids and rigid body, where the fluid predictor output velocity for each fluid particle, while the rigid predictor takes the mean of the signals over all its rigid particles as input, and output a rigid transformation (rotation and translation). The model is trained for 5 epochs. FluidShake. Only one propagation network is used in this environment, and the model is trained for 5 epochs. RiceGrip. Four propagation networks are used due to the hierarchical modeling, and the number of roots for the "rice" is set as 30. The model is trained for 20 epochs. N sample is chosen as 20 for all three cases, where we sample 20 random control sequences, and choose the best performing one as evaluated using our learned model. The evaluation is based on the Chamfer distance between the controlling and the target configuration. FluidShake. In this environment, the control sequence is the speed of the box along the x axis. The method to sample the candidate control sequence is the same as when generating training data of this environment. After selected the best performing control sequence, we first use RMSprop optimizer to optimize the control inputs for 10 iterations using a learning rate of 0.003. We then use model-predictive control to apply the control sequence to the FleX physics engine using Algorithm 1.RiceGrip. In this environment, we need to come up with a sequence of grip configurations, where each grip contains positions, orientation, and closing distance. The method to sample the candidate control sequence is the same as when generating training data of this environment. After selected the best performing control sequence, we first use RMSprop optimizer to optimize the control inputs for 20 iterations using a learning rate of 0.003. We then use model-predictive control to apply the control sequence to the FleX physics engine using Algorithm 1.RiceGrip in Real World. In this environment, we need to come up with a sequence of grip configurations, where each grip contains positions, orientation, and closing distance. The method to sample the candidate control sequence is the same as when generating training data of RiceGrip, and N fill is chosen as 768. Different from the previous case, the physical parameters are always unknown and has to be estimated online. After selected the best performing control sequence, we first use RMSprop optimizer to optimize the control inputs for 20 iterations using a learning rate of 0.003. We then use model-predictive control to apply the control sequence to the real world using Algorithm 1.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

Learning particle dynamics with dynamic interaction graphs for simulating and control rigid bodies, deformable objects, and fluids.

StrokeNet is a novel architecture where the agent is trained to draw by strokes on a differentiable simulation of the environment, which could effectively exploit the power of back-propagation.


Question: Represent the article for finding a single-sentence TLDR

Question: Generative Adversarial Networks (GANs), when trained on large datasets with diverse modes, are known to produce conflated images which do not distinctly belong to any of the modes. We hypothesize that this problem occurs due to the interaction between two facts: For datasets with large variety, it is likely that the modes lie on separate manifolds. The generator (G) is formulated as a continuous function, and the input noise is derived from a connected set, due to which G's output is a connected set. If G covers all modes, then there must be some portion of G's output which connects them. This corresponds to undesirable, conflated images. We develop theoretical arguments to support these intuitions. We propose a novel method to break the second assumption via learnable discontinuities in the latent noise space. Equivalently, it can be viewed as training several generators, thus creating discontinuities in the G function. We also augment the GAN formulation with a classifier C that predicts which noise partition/generator produced the output images, encouraging diversity between each partition/generator. We experiment on MNIST, celebA, STL-10, and a difficult dataset with clearly distinct modes, and show that the noise partitions correspond to different modes of the data distribution, and produce images of superior quality. Generative Adversarial Networks BID8 are powerful generative models that have enjoyed significant attention from the research community in the past few years. Despite several successes, the original formulation for GANs is widely acknowledged to be notoriously difficult to train due to instability issues. In particular, GANs face the mode collapse problem, where the generator resorts to generating a handful of samples which are assigned high probability by the discriminator. Several methods have been introduced to fix the mode collapse problem. BID3,, BID9, BID15, BID21 Despite improvements, state-of-art GANs still fail to generate meaningful samples on diverse and complex datasets such as ImageNet BID5. GANs trained on such datasets produce conflated images which do not distinctly belong to any of the modes present in the dataset. We hypothesize that this problem occurs due to the continuous nature of the generator function, along with the connectedness of the latent noise space, due to which the output set of the generator is also connected. This poses a problem when dealing with complex real life datasets with varied modes. Strong empirical and theoretical evidence suggests that real life images lie on lowdimensional manifolds BID17. It is highly probable that distinct modes (say bedroom images and human face images) lie on disjoint manifolds. If we assume that the generator does not suffer from the mode dropping problem, it must cover all these manifolds in its output. However, the output set being connected, must contain parts which do not belong to any of the manifolds, but simply join them. We refer to such parts of the output as tunnels, since they connect otherwise disjoint manifolds. Tunnels do not resemble any of the images in the dataset, and are not similar to any of the modes. They correspond to the conflated images produced by the generator, and are undesirable. By this reasoning, we suggest that GANs with continuous generators and connected latent noise sets must suffer either from a certain degree of mode dropping or from producing conflated, garbled outputs when trained on complex and varied datasets like ImageNet. We develop methods that allow GANs to cover disjoint manifolds without the use of tunnels, while not compromising on mode coverage. Our approach is to create learnable discontinuities in the latent noise space. This is done by learning N different linear mappings (partitions) in the input layer of the generator. A noise vector (sampled from the standard normal distribution), gets mapped to N different vectors by the input layer, and the rest of the processing remains the same as in standard generators. The output set of each mapping is a connected set, but the union of the N output sets could potentially be disconnected. Thus, we break the connectedness assumption leading to the existence of tunnels. To facilitate learning distinct modes by each partition, we introduce a classifier that predicts which partition created a given input. We modify the loss functions to adjust for this change. We experiment on standard datasets: MNIST , celebA BID14, STL-10 (a subset of ImageNet) BID4, and a tough artificial dataset with very distinct modes -an equal mixture of LSUN BID22 bedrooms and celebA, to verify the efficacy of our method. We compare our with one of the best performing GAN variant BID9, and show an improvement in quality. The major contributions of the paper are summarized below:1. We identify a key problem with training GANs on large & diverse datasets, and provide intuition to explain its cause 2. We develop theoretical analyses to support and introduce rigor in the intuitions provided 3. Motivated by these analyses, we introduce a novel GAN setup to alleviate the problem 4. We experiment on a variety of standard datasets and report improvements over state-of-art formulations 2 RELATED WORK BID8 formulated GAN as a minimax game between two neural networks: generator G θ and discriminator D φ. G θ takes a random noise vector z as input and generates sample G θ (z), while D φ identifies whether input sample is real or generated by the generator G θ. Both G θ and D φ play a two-player minimax game with value function V (G, D): DISPLAYFORM0 where P r (x) is the real data distribution, and P(z) is arbitrary noise distribution (typically uniform or normal distribution). In practice, training GANs using above formulation is highly unstable and requires careful balance of generator and discriminator updates. BID19 proposed a class of CNNs called DCGANs (Deep Convolutional GANs) with certain architectural specifications, and demonstrated better image quality than non-convolutional vanilla GAN architecture. BID6 used Laplacian pyramid framework for the generator, where a separate generative convnet model is trained using GAN approach at each level of pyramid, to generate images in coarse-to-fine fashion. Despite better architectures, GANs suffered from problems like unstable training, vanishing gradients of generator, mode collapse. BID20 proposed several heuristics such as feature matching, minibatch discrimination, historical averaging, label smoothing, primarily to stabilize GAN training. BID3 observed that GAN training can push probability mass in wrong direction, hence are prone to missing modes of data. They proposed regularization techniques to stabilize GAN training and alleviate mode missing problem by fair distribution of probability mass across modes of the real data distribution. BID1 provided theoretical analysis of training dynamics of GANs, and problems including instability and saturation. They revealed fundamental problems with original GAN formulation and provided directions towards solving them. Several papers proposed alternative objective function of generator and discriminator., BID9 proposed new loss function which approximately minimizes Wasserstein distance between real and generated data distribution instead of Jensen Shannon Divergence. They claim their formulation does not require careful balance between generator and discriminator updates, thus lead to stable training without saturating the gradients. They observed no evidence of mode collapse in their experiments. BID15 used squared-loss instead of log-loss in original formulation, which provides generator with better non-vanishing gradients. BID23 view discriminator as an energy function making it possible to use additional loss functions other than logistic output binary classifier, which was found to stabilize GAN training. BID21 propose to train discriminator based on linear separability between hidden representation of real and generated samples and train generator based on decision hyperplanes between hidden representations computed using Linear Discriminant Analysis. For labelled datasets, BID16, BID18 employed label conditioning in both generator and discriminator to generate discriminable and diverse samples across classes. While this helps produce better samples for complex datasets, it requires the presence of labelled data. In this paper we propose methods to improve performance of GANs on complex datasets without making use of labels. In this section, we further develop the ideas from the introduction. We also provide theoretical analyses to lend support to these ideas. Theoretical analyses and empirical studies suggest that probability distributions of real images (denoted by P r) have supports that lie on low dimensional manifolds BID1, BID17 ).We choose to represent the support set S of distribution P r by the set of its connected components, i.e. the set of maximal connected subsets of S. These components are disjoint and their union is S. In other words, they form a partition of S. As suggested earlier, each component is a lowdimensional manifold in high-dimensional space. Throughout this paper, we use the terms manifold of S and connected component of S interchangeably, unless mentioned otherwise. Consider highly distinct images m 1, m 2 of a complex and varied dataset. These images cannot be path-connected, since that would imply the existence of a smooth sequence of images in S starting from m 1, and slowly transitioning to m 2. Such a sequence clearly does not exist for very distinct m 1, m 2, e.g. in a dataset consisting of face and bedroom images, if m 1 is a face, and m 2 is a bedroom, it is not possible for a sequence of realistic facial and bedroom images to smoothly transition from m 1 to m 2. Since path-connectedness and connectedness are equivalent properties in open subsets of R n, the manifolds on which m 1, m 2 lie, must be disconnected, and hence must be separate components of S.We summarize the above discussion with the following :Result 1. Sufficiently distinct images from the support set of real-life image probability distributions must lie on disconnected manifolds. As a corollary, we have:Result 2. The support set of a real-life image probability distribution must have at least as many connected components as the size of the maximal set of (sufficiently, pairwise) distinct modes. The generator is often formulated as a continuous function of the input latent noise vector. Moreover, the probability distribution from which the noise vector is drawn is usually a simple distribution like Gaussian or uniform, which have connected support sets. Finally, we observe that a continuous function, applied to a connected set in an output set which is connected. Thus we have:Result 3. The output set produced by a continuous generator acting on a latent noise distribution with connected support set must be connected. From the previous discussions, we know that the output of the generator must be connected, while diverse distributions must lie on supports consisting of several disconnected manifolds. If the generator does not suffer from mode dropping, then it must cover all the distinct modes in its output set. However, this implies the existence of parts in the generator's output which connect these disconnected manifolds. We call such parts of the output space as tunnels. Since tunnels do not exist in the real distribution, they must correspond to unrealistic images. We condense these ideas to the following : Result 4. A continuous generator, with inputs drawn from a connected set, must either suffer from significant mode dropping or generate unrealistic images to some degree. In the rest of the discussion, we assume that mode dropping is not a significant problem. This is a realistic assumption due to several heuristics and formulations that alleviate this problem BID20, BID3,, BID9, BID21 ). We concentrate on the problem of generation of unrealistic, distorted outputs. If we assume that the generator is K−Lipschitz (this happens with a variety of regularization terms added to the generator's loss), i.e. for any two points z 1, z 2 in the latent noise space we have DISPLAYFORM0 then the generator's output must gradually shift from one manifold to another as we travel in Z space, since the slope is bounded. The region of shift does not belong to any outputs in the real distribution. It is simple to see that some measure of the probability distribution Z is wasted on mapping these unwanted outputs. To demonstrate this, we consider the simple case where P r consists of equal measure of samples of type A and type B, both of which are disconnected and highly distinct from each other. For simplicity, we assume that the latent noise z is drawn from a 1-D region [−1, 1], with uniform probability. Let the distance between set A and set B (defined as inf a∈A,b∈B a − b) be β (> 0 due to high distinction between the sets). Let M A, M B be subsets of Z mapping to A and B respectively. For any arbitrary z 1 ∈ M A and z 2 ∈ M B, from the Lipschitz condition, we have: DISPLAYFORM1 Hence, the distance between any two points of M A and M B must be at least β K, as a of which, the distance between the sets M A and M B is at least β K. Clearly, in our hypothetical case, this in a gap of at least β K wherever M A ends and M B begins. Since we have assumed uniform distribution, a probability measure of β 2K is lost to undesired outputs. It is well-known that conditional GANs BID16, BID18 ) are better at learning complex datasets. The label information incorporated in the inputs of the generator plays a large role in making better outputs. However, we also believe that the discontinuity introduced by the one-hot label representations contributes significantly to improving performance. More concretely, the input to the generator is a latent noise vector, along with the one-hot label representation. Hence, the input space is partitioned into n (number of classes) disconnected components due to the discrete nature of the one-hot labels. This breaks the connectedness assumption of the input space, which was a crucial part of the problem's cause. Note however that conditional GANs require labeled inputs, which may not be available for many datasets. Central to the in section 3 were the assumptions of continuity of the generator and the connectedness of the support set of the latent noise distribution. Breaking either of these assumptions could lead to potential solutions. We now describe novel GAN formulations that breaks these assumptions while not requiring any labeled data. We create trainable discontinuities in the latent noise space by learning N different linear mappings {L 1, L 2 . . . L N} in the input layer of the generator. A noise vector z gets mapped to N vectors {y 1 = L 1 (z)... y N = L N (z)} by the input layer, and the rest of the processing remains the same as in standard generators. We end with N outputs DISPLAYFORM0 Each of the linear layers L i maps the input latent noise space Z to a connected output set O i. While O i is a connected set, the union of the N output sets (O 1 . . . O N) could potentially be disconnected (or each mapping could collapse to the same matrix, if required by the data).This approach of partitioning the noise space can also be seen as training N different generators, with shared parameters, except for the input layer. In this view, the generator function has become potentially discontinuous due to indexability of the outputs (i.e. choosing which output to take).In either view, we have broken one of the assumptions leading to the existence of tunnels. Finally, to facilitate the learning of distinct modes by each partition (or generator), we introduce a classifier C that predicts which partition (or generator) created a given input. We modify the GAN value function to suitably account for this change. We would like to emphasize that this formulation is generic, and can be plugged in with different types of generators, discriminators, partition mappings, and classifiers. Moreover, any improvements in GAN formulations can be independently incorporated into our setup. However, we do make specific design choices for the purpose of this paper, which we describe below. To reduce the cost of training, we take C to be a linear mapping operating on the last hidden layer of the discriminator. We believe that the discriminator extracts useful abstract features for judging the samples produced by the generator, and these can be effectively reused for partition classification. We add the partition classification loss to the existing generator and discriminator losses from the chosen base formulation. The new loss functions are: DISPLAYFORM0 L D, L G are the original loss functions for the discriminator and generator respectively, in the base GAN formulation. For vanilla GANs, DISPLAYFORM1 Here L c (y, C(x)) is the cross-entropy classification loss for input image x, which was generated by partition y, and C(x) is the classifier's softmax output vector.α is a hyperparameter introduced to control the relative importance of generating good samples w.r.t encouraging diversity between the outputs of different partitions. Finally, we must describe the exact meaning of P g in our formulation, since there are N generators. We sample the generator at each training step uniformly, thus making P g = 1 N N i=1 P gi, where P gi is the probability distribution induced by generator i. We now propose a simpler method for implementing a softer version of disconnectedness for the latent noise distribution. In this method, the noise vectors are drawn from a mixture-of-Gaussians with trainable means (µ 1, . . ., µ N) and covariance matrices FIG0,..., Diag(σ N)). If required by the data, the means can move sufficiently far away from each other during training, leading to an almost disconnected distribution. Each Gaussian is given equal probability weight, i.e., it is chosen uniformly at random. Our implementation makes use of the reparameterization trick BID12, BID7 ). We sample z ∼ N (0, I), along with an index i ∼ U nif orm({1, . . ., N}). We then usê z = z σ i + µ i as the latent vector for generating the sample. We let the gradients backpropagate to µ i and σ i, allowing for a learnable latent distribution. Additionally, as mentioned before, we may use a classifier to predict the Gaussian that generated a given sample, hence encouraging diversity between different Gaussians. We experiment with the image generation task on MNIST, celebA consisting of facial images, STL-10, and an artificial dataset called CelebRoom consisting of 100, 000 images randomly sampled from celebA and 100, 000 images randomly sampled from LSUN bedroom dataset. CelebRoom was constructed with the explicit goal of including diverse modes, making it difficult for GANs to train. As in BID9, we present on a toy dataset sampled from an equal mixture 8 bivariate Gaussians. The means are arranged uniformly in a circle with radius 2, and the covariance matrices are set to 0.02I. We modified a popular TensorFlow BID0 implementation of the WGAN-GP architecture 1 for all experiments. N is fixed for each dataset. We use N = 10 for MNIST, and N = 8 for celebA, STL-10, and CelebRoom.α is fixed to 1.0 for all experiments with multiple partitions. We use the ADAM optimizer BID11 for all experiments. For each dataset (except MNIST), we compare the samples obtained using the following setups:For MNIST and celebRoom, we also present generated from the mixture-of-Gaussians latent distribution method. We used the hyperparameters suggested by the original papers for each architecture. The ResNet BID10 architecture used has 4 residual blocks, as described in BID9.We avoid intensive hyperparameter tuning in order to encourage discovery of methods with relatively high hyperparameter insensitivity. We also tried training multiple partitions with the Least Squares GAN BID15 formulation, but noticed that the training quickly diverged. Hence we do not report these here. We present samples on the five datasets. For samples generated using multiple partitions, each column represents the images produced by a single partition. We observe that Vanilla GAN is unable to cover all Gaussians. As training proceeds, it shuttles between covering different subsets of Gaussians. WGAN-GP takes a long time to converge. It is finally able to cover all Gaussians, but not satisfactorily, with several samples lying between different means. We note that by reducing the gradient penalty significantly, we are able to achieve faster convergence. However, we chose to use the recommended hyperparameters for all formulations. Vanilla GAN with partitions is able to cover all the modes, and converges quickly. We observe a slight improvement in quality in the samples generated by our model when using the partition classifier. The images produced with the classifier formulation are more distinct, sharper, and do not contain spurious appendages. We expected each partition to produce a single type of digit, but this does not seem to be the case. Despite this, we do note that the partition classifier's loss was very close to zero upon successful training. This signifies that the partitions did learn to map distinct types of outputs, but not by digit label. This can be explained by the fact that labels are not the only way to classify digits. There are several combinations of digit style, tilt, width etc. that could be captured by the partitions. We observed slight improvements in quality in the samples generated by our model over the WGAN + Gradient Penalty baseline with the DCGAN architecture. FIG0: 64 × 64 celebRoom samples generated using DCGAN architecture with mixture-ofGaussians latent distribution using: no prediction of Gaussians (left), prediction of Gaussians (right)Each partition seems to produce similar types of images, both with and without a partition classifier formulation to encourage diversity. For instance, the fifth partition (fifth column) in DCGAN architecture and WGAN-GP loss seems to capture the mode of smiling women, while the eigth partition (last column) seems to generate faces of men (often bespectacled).We also note that the ResNet architecture is unable to train successfully with partitions. We notice heavy mode collapse, and noisy outputs. Our experiments show heavy conflation of modes in the outputs generated by all GAN formulations. However, this problem is ameliorated to some extent with our partition GAN.In particular, we notice some partitions capturing distinctly well-formed modes. The DCGAN architecture with WGAN-GP loss with 8 partitions shows several such examples. The first partition (first column) seems to create images of vehicles, while the sixth partition (sixth column) seems to generate images of oceans and open skies (along with boats, ships, and airplanes). Similar clustering can also be observed with the ResNet architecture with WGAN-GP loss, where partition four (column four) generates images of birds and planes, while the sixth partition (column six) generates images of animals. We observe significant conflation of bedrooms and facial images in this dataset with the baseline model. Our model alleviates this problem to some extent, but does not solve it completely. However, we see that each partition does either primarily create faces or bedrooms. We would like to note that if a particular partition generates bad quality samples, it might be due to the inherent difficulty in creating that portion of the real distribution. In such cases, we can drop the outputs from that partition at the cost of not capturing 1 N of the distribution. We highlighted a major problem in training GANs on complex image datasets and introduced theoretical analysis for the problem of generation of unrealistic, conflated images in such cases. We proposed the addition of discontinuity in latent noise space of the generator for covering disjoint and diverse modes of the data distribution, and augmented the loss functions to encourage diversity. We showed improvements over existing models without much hyperparameter tuning. In future, we hope to perform an extensive exploration of the search space to obtain a set of hyperparameters along with better methods to introduce discontinuities in the generator that perform well on a variety of datasets, while significantly improving image quality.

Represent the TLDR for finding the corresponding scientific article

We introduce theory to explain the failure of GANs on complex datasets and propose a solution to fix it.

We leverage deterministic autoencoders as generative models by proposing mixing functions which combine hidden states from pairs of images. These mixes are made to look realistic through an adversarial framework.