While these numerical methods have been fruitful, a closed form expression for the MMSE estimator is nevertheless possible if we are willing to make some compromises. One possibility is to abandon the full optimality requirements and seek a technique minimizing the MSE within a particular class of estimators, such as the class of linear estimators. Thus, we postulate that the conditional expectation of given is a simple linear function of , , where the measurement is a random vector, is a matrix and is a vector. This can be seen as the first order Taylor approximation of . The linear MMSE estimator is the estimator achieving minimum MSE among all estimators of such form. That is, it solves the following the optimization problem: One advantage of such linear MMSE estimator is that it is not necessary to explicitly calculate the posterior probability density function of . Such linear estimator only depends on the first two moments of and . So although it may be convenient to assume that and are jointly Gaussian, it is not necessary to make this assumption, so long as the assumed distribution has well defined first and second moments. The form of the linear estimator does not depend on the type of the assumed underlying distribution. The expression for optimal and is given by: where , the is cross-covariance matrix between and , the is auto-covariance matrix of . Thus, the expression for linear MMSE estimator, its mean, and its auto-covariance is given by where the is cross-covariance matrix between and . Lastly, the error covariance and minimum mean square error achievable by such estimator is Let us have the optimal linear MMSE estimator given as , where we are required to find the expression for and . It is required that the MMSE estimator be unbiased. This means, Plugging the expression for in above, we get where and . Thus we can re-write the estimator as and the expression for estimation error becomes From the orthogonality principle, we can have , where we take . Here the left-hand-side term is When equated to zero, we obtain the desired expression for as The is cross-covariance matrix between X and Y, and is auto-covariance matrix of Y. Since , the expression can also be re-written in terms of as Thus the full expression for the linear MMSE estimator is Since the estimate is itself a random variable