Eigen Decomposition of Data Matrix for PCA
Some Intro

Eigen-Decompostion does lots of things among them one thing is interesting. Eigen-Decomposition usually captures the characteristic of the matrix it is applied to. For instance is the matrix is the matrix of similarities, the result of eigen decomposition will be a clustering (see Spectral Clustering).

PCA

The idea behind PCA is to find coordinates on which the intrinsic variance of data is maximized. Well ... according to Intro, if I have a matrix encoding joint variance (co-variance) of all pairs of variables, then I can use eigen decomposition to simply capture this property for me! This is what PCA does.