Title: Copula Variational Bayes inference via information geometry,
Abstract: Variational Bayes (VB), also known as independent mean-field approximation,
has become a popular method for Bayesian network inference in recent years. Its
application is vast, e.g. in neural network, compressed sensing, clustering,
etc. to name just a few. In this paper, the independence constraint in VB will
be relaxed to a conditional constraint class, called copula in statistics.
Since a joint probability distribution always belongs to a copula class, the
novel copula VB (CVB) approximation is a generalized form of VB. Via
information geometry, we will see that CVB algorithm iteratively projects the
original joint distribution to a copula constraint space until it reaches a
local minimum Kullback-Leibler (KL) divergence. By this way, all mean-field
approximations, e.g. iterative VB, Expectation-Maximization (EM), Iterated
Conditional Mode (ICM) and k-means algorithms, are special cases of CVB
approximation.
For a generic Bayesian network, an augmented hierarchy form of CVB will also
be designed. While mean-field algorithms can only return a locally optimal
approximation for a correlated network, the augmented CVB network, which is an
optimally weighted average of a mixture of simpler network structures, can
potentially achieve the globally optimal approximation for the first time. Via
simulations of Gaussian mixture clustering, the classification's accuracy of
CVB will be shown to be far superior to that of state-of-the-art VB, EM and
k-means algorithms.