Title: Heteroskedastic PCA: Algorithm, Optimality, and Applications,
Abstract: Principal component analysis (PCA) and singular value decomposition (SVD) are
widely used in statistics, machine learning, and applied mathematics. It has
been well studied in the case of homoskedastic noise, where the noise levels of
the contamination are homogeneous.
In this paper, we consider PCA and SVD in the presence of heteroskedastic
noise, which arises naturally in a range of applications. We introduce a
general framework for heteroskedastic PCA and propose an algorithm called
HeteroPCA, which involves iteratively imputing the diagonal entries to remove
the bias due to heteroskedasticity. This procedure is computationally efficient
and provably optimal under the generalized spiked covariance model. A key
technical step is a deterministic robust perturbation analysis on the singular
subspace, which can be of independent interest. The effectiveness of the
proposed algorithm is demonstrated in a suite of applications, including
heteroskedastic low-rank matrix denoising, Poisson PCA, and SVD based on
heteroskedastic and incomplete data.