Title: Geometry of Factored Nuclear Norm Regularization,
Abstract: This work investigates the geometry of a nonconvex reformulation of
minimizing a general convex loss function $f(X)$ regularized by the matrix
nuclear norm $\|X\|_*$. Nuclear-norm regularized matrix inverse problems are at
the heart of many applications in machine learning, signal processing, and
control. The statistical performance of nuclear norm regularization has been
studied extensively in literature using convex analysis techniques. Despite its
optimal performance, the resulting optimization has high computational
complexity when solved using standard or even tailored fast convex solvers. To
develop faster and more scalable algorithms, we follow the proposal of
Burer-Monteiro to factor the matrix variable $X$ into the product of two
smaller rectangular matrices $X=UV^T$ and also replace the nuclear norm
$\|X\|_*$ with $(\|U\|_F^2+\|V\|_F^2)/2$. In spite of the nonconvexity of the
factored formulation, we prove that when the convex loss function $f(X)$ is
$(2r,4r)$-restricted well-conditioned, each critical point of the factored
problem either corresponds to the optimal solution $X^\star$ of the original
convex optimization or is a strict saddle point where the Hessian matrix has a
strictly negative eigenvalue. Such a geometric structure of the factored
formulation allows many local search algorithms to converge to the global
optimum with random initializations.