of the measurement . The estimation error vector is given by and its mean squared error (MSE) is given by the trace of error covariance matrix where the expectation is taken over conditioned on . When is a scalar variable, the MSE expression simplifies to . Note that MSE can equivalently be defined in other ways, since The MMSE estimator is then defined as the estimator achieving minimal MSE: Properties When the means and variances are finite, the MMSE estimator is uniquely defined and is given by: In other words, the MMSE estimator is the conditional expectation of given the known observed value of the measurements. Also, since is the posterior mean, the error covariance matrix is equal to the posterior covariance matrix, . The MMSE estimator is unbiased (under the regularity assumptions mentioned above): The MMSE estimator is asymptotically unbiased and it converges in distribution to the normal distribution: where is the Fisher information of . Thus, the MMSE estimator is asymptotically efficient. The orthogonality principle: When is a scalar, an estimator constrained to be of certain form is an optimal estimator, i.e. if and only if for all in closed, linear subspace of the measurements. For random vectors, since the MSE for estimation of a random vector is the sum of the MSEs of the coordinates, finding the MMSE estimator of a random vector decomposes into finding the MMSE estimators of the coordinates of X separately: for all i and j. More succinctly put, the cross-correlation between the minimum estimation error and the estimator should be zero, If and are jointly Gaussian, then the MMSE estimator is linear, i.e., it has the form for matrix and constant . This can be directly shown using the Bayes theorem. As a consequence, to find the MMSE estimator, it is sufficient to find the linear MMSE estimator. Linear MMSE estimator In many cases, it is not possible to determine the analytical expression of the MMSE estimator. Two basic numerical approaches to obtain the MMSE estimate depends on either finding the conditional expectation or finding the minima of MSE. Direct numerical evaluation of the conditional expectation is computationally expensive since it often requires multidimensional integration usually done via Monte Carlo methods. Another computational approach is to directly seek the minima of the MSE using techniques such as the stochastic gradient descent methods ; but this method still requires the evaluation of expectation.