and as well as an indirect circular dependence on , and through and . This suggests an iterative procedure that alternates between two steps: An E-step that computes the value of using the current values of all the other parameters. An M-step that uses the new value of to compute new values of all the other parameters. Note that these steps correspond closely with the standard EM algorithm to derive a maximum likelihood or maximum a posteriori (MAP) solution for the parameters of a Gaussian mixture model. The responsibilities in the E step correspond closely to the posterior probabilities of the latent variables given the data, i.e. ; the computation of the statistics , , and corresponds closely to the computation of corresponding "soft-count" statistics over the data; and the use of those statistics to compute new values of the parameters corresponds closely to the use of soft counts to compute new parameter values in normal EM over a Gaussian mixture model. Exponential-family distributions Note that in the previous example, once the distribution over unobserved variables was assumed to factorize into distributions over the "parameters" and distributions over the "latent data", the derived "best" distribution for each variable was in the same family as the corresponding prior distribution over the variable. This is a general result that holds true for all prior distributions derived from the exponential family. See also Variational message passing: a modular algorithm for variational Bayesian inference. Variational autoencoder: an artificial neural network belonging to the families of probabilistic graphical models and Variational Bayesian methods. Expectation-maximization algorithm: a related approach which corresponds to a special case of variational Bayesian inference. Generalized filtering: a variational filtering scheme for nonlinear state space models. Calculus of variations: the field of mathematical analysis that deals with maximizing or minimizing functionals. Maximum entropy discrimination: This is a variational inference framework that allows for introducing and accounting for additional large-margin constraints References External links The on-line textbook: Information Theory, Inference, and Learning Algorithms, by David J.C. MacKay provides an introduction to variational methods (p. 422). A Tutorial on Variational Bayes. Fox, C. and Roberts, S. 2012. Artificial Intelligence Review, . Variational-Bayes Repository A repository of research papers, software, and links related to the use of variational methods for approximate Bayesian learning up to 2003. Variational Algorithms for Approximate Bayesian Inference, by M. J. Beal includes comparisons of EM to Variational Bayesian EM and