the covariance matrix C, and Matrix V, also of dimension p × p, contains p column vectors, each of length p, which represent the p eigenvectors of the covariance matrix C. The eigenvalues and eigenvectors are ordered and paired. The jth eigenvalue corresponds to the jth eigenvector. Matrix V denotes the matrix of right eigenvectors (as opposed to left eigenvectors). In general, the matrix of right eigenvectors need not be the (conjugate) transpose of the matrix of left eigenvectors. Rearrange the eigenvectors and eigenvalues Sort the columns of the eigenvector matrix V and eigenvalue matrix D in order of decreasing eigenvalue. Make sure to maintain the correct pairings between the columns in each matrix. Compute the cumulative energy content for each eigenvector The eigenvalues represent the distribution of the source data's energy among each of the eigenvectors, where the eigenvectors form a basis for the data. The cumulative energy content g for the jth eigenvector is the sum of the energy content across all of the eigenvalues from 1 through j: Select a subset of the eigenvectors as basis vectors Save the first L columns of V as the p × L matrix W: where Use the vector g as a guide in choosing an appropriate value for L. The goal is to choose a value of L as small as possible while achieving a reasonably high value of g on a percentage basis. For example, you may want to choose L so that the cumulative energy g is above a certain threshold, like 90 percent. In this case, choose the smallest value of L such that Project the data onto the new basis The projected data points are the rows of the matrix That is, the first column of is the projection of the data points onto the first principal component, the second column is the projection onto the second principal component, etc. Derivation of PCA using the covariance method Let X be a d-dimensional random vector expressed as column vector. Without loss of generality, assume X has zero mean. We want to find a orthonormal transformation matrix P so that PX has a diagonal covariance matrix (that is, PX is a random vector with all its distinct components pairwise uncorrelated). A quick computation assuming were unitary yields: Hence holds if and only if were diagonalisable by . This is very constructive, as cov(X) is guaranteed to be a