n-by-1 known column vector whose values can change with time, is n-by-1 random column vector to be estimated, and is scalar noise term with variance . After (k+1)-th observation, the direct use of above recursive equations give the expression for the estimate as: where is the new scalar observation and the gain factor is n-by-1 column vector given by The is n-by-n error covariance matrix given by Here, no matrix inversion is required. Also, the gain factor, , depends on our confidence in the new data sample, as measured by the noise variance, versus that in the previous data. The initial values of and are taken to be the mean and covariance of the aprior probability density function of . Alternative approaches: This important special case has also given rise to many other iterative methods (or adaptive filters), such as the least mean squares filter and recursive least squares filter, that directly solves the original MSE optimization problem using stochastic gradient descents. However, since the estimation error cannot be directly observed, these methods try to minimize the mean squared prediction error . For instance, in the case of scalar observations, we have the gradient Thus, the update equation for the least mean square filter is given by where is the scalar step size and the expectation is approximated by the instantaneous value . As we can see, these methods bypass the need for covariance matrices. Examples Example 1 We shall take a linear prediction problem as an example. Let a linear combination of observed scalar random variables and be used to estimate another future scalar random variable such that . If the random variables are real Gaussian random variables with zero mean and its covariance matrix given by then our task is to find the coefficients such that it will yield an optimal linear estimate . In terms of the terminology developed in the previous sections, for this problem we have the observation vector , the estimator matrix as a row vector, and the estimated variable as a scalar quantity. The autocorrelation matrix is defined as The cross correlation matrix is defined as We now solve the equation by inverting and pre-multiplying to get So we have and as the optimal coefficients for . Computing the minimum mean square error then gives . Note that it is not necessary to obtain an explicit matrix inverse of to compute the value