Title: Wasserstein Distributional Robustness and Regularization in Statistical Learning,
Abstract: A central question in statistical learning is to design algorithms that not
only perform well on training data, but also generalize to new and unseen data.
In this paper, we tackle this question by formulating a distributionally robust
stochastic optimization (DRSO) problem, which seeks a solution that minimizes
the worst-case expected loss over a family of distributions that are close to
the empirical distribution in Wasserstein distances. We establish a connection
between such Wasserstein DRSO and regularization. More precisely, we identify a
broad class of loss functions, for which the Wasserstein DRSO is asymptotically
equivalent to a regularization problem with a gradient-norm penalty. Such
relation provides new interpretations for problems involving regularization,
including a great number of statistical learning problems and discrete choice
models (e.g. multinomial logit). The connection suggests a principled way to
regularize high-dimensional, non-convex problems. This is demonstrated through
the training of Wasserstein generative adversarial networks in deep learning.