Title: A Unifying Framework for Convergence Analysis of Approximate Newton Methods,
Abstract: Many machine learning models are reformulated as optimization problems. Thus,
it is important to solve a large-scale optimization problem in big data
applications. Recently, subsampled Newton methods have emerged to attract much
attention for optimization due to their efficiency at each iteration, rectified
a weakness in the ordinary Newton method of suffering a high cost in each
iteration while commanding a high convergence rate. Other efficient stochastic
second order methods are also proposed. However, the convergence properties of
these methods are still not well understood. There are also several important
gaps between the current convergence theory and the performance in real
applications. In this paper, we aim to fill these gaps. We propose a unifying
framework to analyze local convergence properties of second order methods.
Based on this framework, our theoretical analysis matches the performance in
real applications.