the sake of brevity. Exponentiating both sides, we can see that is a gamma distribution. Specifically: Algorithm for computing the parameters Let us recap the conclusions from the previous sections: and In each case, the parameters for the distribution over one of the variables depend on expectations taken with respect to the other variable. We can expand the expectations, using the standard formulas for the expectations of moments of the Gaussian and gamma distributions: Applying these formulas to the above equations is trivial in most cases, but the equation for takes more work: We can then write the parameter equations as follows, without any expectations: Note that there are circular dependencies among the formulas for and . This naturally suggests an EM-like algorithm: Compute and Use these values to compute and Initialize to some arbitrary value. Use the current value of along with the known values of the other parameters, to compute . Use the current value of along with the known values of the other parameters, to compute . Repeat the last two steps until convergence (i.e. until neither value has changed more than some small amount). We then have values for the hyperparameters of the approximating distributions of the posterior parameters, which we can use to compute any properties we want of the posterior — e.g. its mean and variance, a 95% highest-density region (the smallest interval that includes 95% of the total probability), etc. It can be shown that this algorithm is guaranteed to converge to a local maximum. Note also that the posterior distributions have the same form as the corresponding prior distributions. We did not assume this; the only assumption we made was that the distributions factorize, and the form of the distributions followed naturally. It turns out (see below) that the fact that the posterior distributions have the same form as the prior distributions is not a coincidence, but a general result whenever the prior distributions are members of the exponential family, which is the case for most of the standard distributions. Further discussion Step-by-step recipe The above example shows the method by which the variational-Bayesian approximation to a posterior probability density in a given Bayesian network is derived: Describe the network with a graphical model, identifying the observed variables (data) and unobserved variables (parameters and latent variables ) and their conditional probability distributions. Variational Bayes will then construct an approximation to the posterior