non-negative definite matrix and thus is guaranteed to be diagonalisable by some unitary matrix. Covariance-free computation In practical implementations, especially with high dimensional data (large ), the naive covariance method is rarely used because it is not efficient due to high computational and memory costs of explicitly determining the covariance matrix. The covariance-free approach avoids the operations of explicitly calculating and storing the covariance matrix , instead utilizing one of matrix-free methods, for example, based on the function evaluating the product at the cost of operations. Iterative computation One way to compute the first principal component efficiently is shown in the following pseudo-code, for a data matrix with zero mean, without ever computing its covariance matrix. = a random vector of length r = r / norm(r) do times: (a vector of length ) return This power iteration algorithm simply calculates the vector , normalizes, and places the result back in . The eigenvalue is approximated by , which is the Rayleigh quotient on the unit vector for the covariance matrix . If the largest singular value is well separated from the next largest one, the vector gets close to the first principal component of within the number of iterations , which is small relative to , at the total cost . The power iteration convergence can be accelerated without noticeably sacrificing the small cost per iteration using more advanced matrix-free methods, such as the Lanczos algorithm or the Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) method. Subsequent principal components can be computed one-by-one via deflation or simultaneously as a block. In the former approach, imprecisions in already computed approximate principal components additively affect the accuracy of the subsequently computed principal components, thus increasing the error with every new computation. The latter approach in the block power method replaces single-vectors and with block-vectors, matrices and . Every column of approximates one of the leading principal components, while all columns are iterated simultaneously. The main calculation is evaluation of the product . Implemented, for example, in LOBPCG, efficient blocking eliminates the accumulation of the errors, allows using high-level BLAS matrix-matrix product functions, and typically leads to faster convergence, compared to the single-vector one-by-one technique. The NIPALS method Non-linear iterative partial least squares (NIPALS) is a variant the classical power iteration with matrix deflation by subtraction implemented for computing the first few components in a principal component or partial least squares analysis.