of this property is that the last few PCs are not simply unstructured left-overs after removing the important PCs. Because these last PCs have variances as small as possible they are useful in their own right. They can help to detect unsuspected near-constant linear relationships between the elements of , and they may also be useful in regression, in selecting a subset of variables from , and in outlier detection. Property 3: (Spectral decomposition of ) Before we look at its usage, we first look at diagonal elements, Then, perhaps the main statistical implication of the result is that not only can we decompose the combined variances of all the elements of into decreasing contributions due to each PC, but we can also decompose the whole covariance matrix into contributions from each PC. Although not strictly decreasing, the elements of will tend to become smaller as increases, as is nonincreasing for increasing , whereas the elements of tend to stay about the same size because of the normalization constraints: . Limitations As noted above, the results of PCA depend on the scaling of the variables. This can be cured by scaling each feature by its standard deviation, so that one ends up with dimensionless features with unital variance. The applicability of PCA as described above is limited by certain (tacit) assumptions made in its derivation. In particular, PCA can capture linear correlations between the features but fails when this assumption is violated (see Figure 6a in the reference). In some cases, coordinate transformations can restore the linearity assumption and PCA can then be applied (see kernel PCA). Another limitation is the mean-removal process before constructing the covariance matrix for PCA. In fields such as astronomy, all the signals are non-negative, and the mean-removal process will force the mean of some astrophysical exposures to be zero, which consequently creates unphysical negative fluxes, and forward modeling has to be performed to recover the true magnitude of the signals. As an alternative method, non-negative matrix factorization focusing only on the non-negative elements in the matrices, which is well-suited for astrophysical observations. See more at Relation between PCA and Non-negative Matrix Factorization. PCA is at a disadvantage if the data has not been standardized before applying the algorithm to it. PCA transforms original data into data that is relevant to the principal components of that data, which means that the new data variables cannot