have implicitly assumed that the statistical properties of does not change with time. In other words, is stationary. In the context of linear MMSE estimator, the formula for the estimate will have the same form as before. However, the mean and covariance matrices of and will need to be replaced by those of the prior density and likelihood , respectively. The mean and covariance matrix of the prior density is given by the previous MMSE estimate, , and the error covariance matrix, respectively, as per by the property of MMSE estimators. Similarly, for the linear observation process, the mean and covariance matrix of the likelihood is given by and . The difference between the predicted value of , as given by , and the observed value of gives the prediction error , which is also referred to as innovation. It is more convenient to represent the linear MMSE in terms of the prediction error, whose mean and covariance are and . Hence, in the estimate update formula, we should replace and by and , respectively. Also, we should replace and by and . Lastly, we replace by Thus, we have the new estimate as and the new error covariance as From the point of view of linear algebra, for sequential estimation, if we have an estimate based on measurements generating space , then after receiving another set of measurements, we should subtract out from these measurements that part that could be anticipated from the result of the first measurements. In other words, the updating must be based on that part of the new data which is orthogonal to the old data. The repeated use of the above two equations as more observations become available lead to recursive estimation techniques. The expressions can be more compactly written as The matrix is often referred to as the Kalman gain factor. The alternative formulation of the above algorithm will give The repetition of these three steps as more data becomes available leads to an iterative estimation algorithm. The generalization of this idea to non-stationary cases gives rise to the Kalman filter. The three update steps outlined above indeed form the update step of the Kalman filter. Special case: scalar observations As an important special case, an easy to use recursive expression can be derived when at each k-th time instant the underlying linear observation process yields a scalar such that , where is