Title: Fast Global Convergence via Landscape of Empirical Loss,
Abstract: While optimizing convex objective (loss) functions has been a powerhouse for
machine learning for at least two decades, non-convex loss functions have
attracted fast growing interests recently, due to many desirable properties
such as superior robustness and classification accuracy, compared with their
convex counterparts. The main obstacle for non-convex estimators is that it is
in general intractable to find the optimal solution. In this paper, we study
the computational issues for some non-convex M-estimators. In particular, we
show that the stochastic variance reduction methods converge to the global
optimal with linear rate, by exploiting the statistical property of the
population loss. En route, we improve the convergence analysis for the batch
gradient method in \cite{mei2016landscape}.