The solution is to only use invertible data augmentation: instead of "randomly rotate the picture by 0, 90, 180, 270 degrees with equal probability", use "randomly rotate the picture by 90, 180, 270 degrees with 0.1 probability, and keep the picture as it is with 0.7 probability". This way, the generator is still rewarded to keep images oriented the same way as un-augmented ImageNet pictures. Abstractly, the effect of randomly sampling transformations from the distribution is to define a Markov kernel . Then, the data-augmented GAN game pushes the generator to find some , such that where is the Markov kernel convolution. A data-augmentation method is defined to be invertible if its Markov kernel satisfiesImmediately by definition, we see that composing multiple invertible data-augmentation methods results in yet another invertible method. Also by definition, if the data-augmentation method is invertible, then using it in a GAN game does not change the optimal strategy for the generator, which is still . There are two prototypical examples of invertible Markov kernels: Discrete case: Invertible stochastic matrices, when is finite. For example, if is the set of four images of an arrow, pointing in 4 directions, and the data augmentation is "randomly rotate the picture by 90, 180, 270 degrees with probability , and keep the picture as it is with probability ", then the Markov kernel can be represented as a stochastic matrix: and is an invertible kernel iff is an invertible matrix, that is, . Continuous case: The gaussian kernel, when for some . For example, if is the space of 256x256 images, and the data-augmentation method is "generate a gaussian noise , then add to the image", then is just convolution by the density function of . This is invertible, because convolution by a gaussian is just convolution by the heat kernel, so given any , the convolved distribution can be obtained by heating up precisely according to , then wait for time . With that, we can recover by running the heat equation backwards in time for . More examples of invertible data augmentations are found in the paper. SinGAN SinGAN pushes data augmentation to the limit, by using only a single image as training data and performing data augmentation on it. The GAN architecture is adapted to this training method by using a multi-scale pipeline. The generator is decomposed into a pyramid of generators , with the lowest