Title: Relaxed Wasserstein with Applications to GANs,
Abstract: We propose a novel class of statistical divergences called \textit{Relaxed
Wasserstein} (RW) divergence. RW divergence generalizes Wasserstein divergence
and is parametrized by a class of strictly convex and differentiable functions.
We establish for RW divergence several probabilistic properties, which are
critical for the success of Wasserstein divergence. In particular, we show that
RW divergence is dominated by Total Variation (TV) and Wasserstein-$L^2$
divergence, and that RW divergence has continuity, differentiability and
duality representation. Finally, we provide a nonasymptotic moment estimate and
a concentration inequality for RW divergence.
Our experiments on the image generation task demonstrate that RW divergence
is a suitable choice for GANs. Indeed, the performance of RWGANs with
Kullback-Leibler (KL) divergence is very competitive with other
state-of-the-art GANs approaches. Furthermore, RWGANs possess better
convergence properties than the existing WGANs with competitive inception
scores. To the best of our knowledge, our new conceptual framework is the first
to not only provide the flexibility in designing effective GANs scheme, but
also the possibility in studying different losses under a unified mathematical
framework.