How close are the eigenvectors and eigenvalues of the sample and actual covariance matrices?
  How many samples are sufficient to guarantee that the eigenvectors and
eigenvalues of the sample covariance matrix are close to those of the actual
covariance matrix? For a wide family of distributions, including distributions
with finite second moment and distributions supported in a centered Euclidean
ball, we prove that the inner product between eigenvectors of the sample and
actual covariance matrices decreases proportionally to the respective
eigenvalue distance. Our findings imply non-asymptotic concentration bounds for
eigenvectors, eigenspaces, and eigenvalues. They also provide conditions for
distinguishing principal components based on a constant number of samples.
