Title: A Statistical Learning Approach to Modal Regression,
Abstract: This paper studies the nonparametric modal regression problem systematically
from a statistical learning view. Originally motivated by pursuing a
theoretical understanding of the maximum correntropy criterion based regression
(MCCR), our study reveals that MCCR with a tending-to-zero scale parameter is
essentially modal regression. We show that nonparametric modal regression
problem can be approached via the classical empirical risk minimization. Some
efforts are then made to develop a framework for analyzing and implementing
modal regression. For instance, the modal regression function is described, the
modal regression risk is defined explicitly and its \textit{Bayes} rule is
characterized; for the sake of computational tractability, the surrogate modal
regression risk, which is termed as the generalization risk in our study, is
introduced. On the theoretical side, the excess modal regression risk, the
excess generalization risk, the function estimation error, and the relations
among the above three quantities are studied rigorously. It turns out that
under mild conditions, function estimation consistency and convergence may be
pursued in modal regression as in vanilla regression protocols, such as mean
regression, median regression, and quantile regression. However, it outperforms
these regression models in terms of robustness as shown in our study from a
re-descending M-estimation view. This coincides with and in return explains the
merits of MCCR on robustness. On the practical side, the implementation issues
of modal regression including the computational algorithm and the tuning
parameters selection are discussed. Numerical assessments on modal regression
are also conducted to verify our findings empirically.