Rayleigh-Schr " o dinger perturbation theory is a well-known theory in quantum mechanics and it offers useful characterization of eigenvectors of a perturbed matrix. Suppose @math and perturbation @math are both Hermitian matrices, @math , @math are eigenvalues of @math in descending order, and @math are leading eigenvectors of @math and @math . Rayleigh-Schr " o dinger theory shows asymptotically, @math where @math . However, the asymptotic theory does not apply to larger @math ; in particular, it fails when @math . In this paper, we present a nonasymptotic theory with @math being a random matrix. We prove that, when @math and @math has independent and centered subgaussian entries above its diagonal, with high probability, for all @math simultaneously, under a condition on eigenvalues of @math that involves all gaps @math . This bound is valid, even in cases where @math . The result is optimal, except for a log term. It also leads to an improvement of Davis-Kahan theorem.
@cite_5: When a Hermitian linear operator is slightly perturbed, by how much can its invariant subspaces change? Given some approximations to a cluster of neighboring eigenvalues and to the corresponding eigenvectors of a real symmetric matrix, and given an estimate for the gap that separates the cluster from all other eigenvalues, how much can the subspace spanned by the eigenvectors differ from the subspace spanned by our approximations? These questions are closely related; both are investigated here. The difference between the two subspaces is characterized in terms of certain angles through which one subspace must be rotated in order most directly to reach the other. These angles unify the treatment of natural geometric, operator-theoretic and error-analytic questions concerning those subspaces. Sharp bounds upon trigonometric functions of these angles are obtained from the gap and from bounds upon either the perturbation (1st question) or a computable residual (2nd question). An example is included.
@cite_6: LetA be anm ×n-matrix which is slightly perturbed. In this paper we will derive an estimate of how much the invariant subspaces ofA H A andAA H will then be affected. These bounds have the sin ϑ theorem for Hermitian linear operators in Davis and Kahan [1] as a special case. They are applicable to computational solution of overdetermined systems of linear equations and especially cover the rank deficient case when the matrix is replaced by one of lower rank.
@cite_11: In statistics and machine learning, people are often interested in the eigenvectors (or singular vectors) of certain matrices (e.g. covariance matrices, data matrices, etc). However, those matrices are usually perturbed by noises or statistical errors, either from random sampling or structural patterns. One usually employs Davis-Kahan @math theorem to bound the difference between the eigenvectors of a matrix @math and those of a perturbed matrix @math , in terms of @math norm. In this paper, we prove that when @math is a low-rank and incoherent matrix, the @math norm perturbation bound of singular vectors (or eigenvectors in the symmetric case) is smaller by a factor of @math or @math for left and right vectors, where @math and @math are the matrix dimensions. The power of this new perturbation result is shown in robust covariance estimation, particularly when random variables have heavy tails. There, we propose new robust covariance estimators and establish their asymptotic properties using the newly developed perturbation bound. Our theoretical results are verified through extensive numerical experiments.
In classical paper by @cite_5 , only deterministic matrices were considered. @cite_6 established similar bounds for singular value decomposition, which is similar to spectral decomposition but applies to non-Hermitian matrices as well. Recently, @cite_11 studied @math perturbation bounds for structured low-rank matrices that have incoherent eigenvectors, but their result is suboptimal when perturbation matrix is random.