Title: Training Generative Adversarial Networks via Primal-Dual Subgradient Methods: A Lagrangian Perspective on GAN,
Abstract: We relate the minimax game of generative adversarial networks (GANs) to
finding the saddle points of the Lagrangian function for a convex optimization
problem, where the discriminator outputs and the distribution of generator
outputs play the roles of primal variables and dual variables, respectively.
This formulation shows the connection between the standard GAN training process
and the primal-dual subgradient methods for convex optimization. The inherent
connection does not only provide a theoretical convergence proof for training
GANs in the function space, but also inspires a novel objective function for
training. The modified objective function forces the distribution of generator
outputs to be updated along the direction according to the primal-dual
subgradient methods. A toy example shows that the proposed method is able to
resolve mode collapse, which in this case cannot be avoided by the standard GAN
or Wasserstein GAN. Experiments on both Gaussian mixture synthetic data and
real-world image datasets demonstrate the performance of the proposed method on
generating diverse samples.