this model in great detail to illustrate the workings of the variational Bayes method. For mathematical convenience, in the following example we work in terms of the precision — i.e. the reciprocal of the variance (or in a multivariate Gaussian, the inverse of the covariance matrix) — rather than the variance itself. (From a theoretical standpoint, precision and variance are equivalent since there is a one-to-one correspondence between the two.) The mathematical model We place conjugate prior distributions on the unknown mean and precision , i.e. the mean also follows a Gaussian distribution while the precision follows a gamma distribution. In other words: The hyperparameters and in the prior distributions are fixed, given values. They can be set to small positive numbers to give broad prior distributions indicating ignorance about the prior distributions of and . We are given data points and our goal is to infer the posterior distribution of the parameters and The joint probability The joint probability of all variables can be rewritten as where the individual factors are where Factorized approximation Assume that , i.e. that the posterior distribution factorizes into independent factors for and . This type of assumption underlies the variational Bayesian method. The true posterior distribution does not in fact factor this way (in fact, in this simple case, it is known to be a Gaussian-gamma distribution), and hence the result we obtain will be an approximation. Derivation of Then In the above derivation, , and refer to values that are constant with respect to . Note that the term is not a function of and will have the same value regardless of the value of . Hence in line 3 we can absorb it into the constant term at the end. We do the same thing in line 7. The last line is simply a quadratic polynomial in . Since this is the logarithm of , we can see that itself is a Gaussian distribution. With a certain amount of tedious math (expanding the squares inside of the braces, separating out and grouping the terms involving and and completing the square over ), we can derive the parameters of the Gaussian distribution: Note that all of the above steps can be shortened by using the formula for the sum of two quadratics. In other words: Derivation of The derivation of is similar to above, although we omit some of the details for