Title: A Classification-Based Study of Covariate Shift in GAN Distributions,
Abstract: A basic, and still largely unanswered, question in the context of Generative
Adversarial Networks (GANs) is whether they are truly able to capture all the
fundamental characteristics of the distributions they are trained on. In
particular, evaluating the diversity of GAN distributions is challenging and
existing methods provide only a partial understanding of this issue. In this
paper, we develop quantitative and scalable tools for assessing the diversity
of GAN distributions. Specifically, we take a classification-based perspective
and view loss of diversity as a form of covariate shift introduced by GANs. We
examine two specific forms of such shift: mode collapse and boundary
distortion. In contrast to prior work, our methods need only minimal human
supervision and can be readily applied to state-of-the-art GANs on large,
canonical datasets. Examining popular GANs using our tools indicates that these
GANs have significant problems in reproducing the more distributional
properties of their training dataset.