Title: On Quadratic Convergence of DC Proximal Newton Algorithm for Nonconvex Sparse Learning in High Dimensions,
Abstract: We propose a DC proximal Newton algorithm for solving nonconvex regularized
sparse learning problems in high dimensions. Our proposed algorithm integrates
the proximal Newton algorithm with multi-stage convex relaxation based on the
difference of convex (DC) programming, and enjoys both strong computational and
statistical guarantees. Specifically, by leveraging a sophisticated
characterization of sparse modeling structures/assumptions (i.e., local
restricted strong convexity and Hessian smoothness), we prove that within each
stage of convex relaxation, our proposed algorithm achieves (local) quadratic
convergence, and eventually obtains a sparse approximate local optimum with
optimal statistical properties after only a few convex relaxations. Numerical
experiments are provided to support our theory.