A Variational Approach for Approximating Bayesian Networks by Edge
  Deletion
  We consider in this paper the formulation of approximate inference in
Bayesian networks as a problem of exact inference on an approximate network
that results from deleting edges (to reduce treewidth). We have shown in
earlier work that deleting edges calls for introducing auxiliary network
parameters to compensate for lost dependencies, and proposed intuitive
conditions for determining these parameters. We have also shown that our method
corresponds to IBP when enough edges are deleted to yield a polytree, and
corresponds to some generalizations of IBP when fewer edges are deleted. In
this paper, we propose a different criteria for determining auxiliary
parameters based on optimizing the KL-divergence between the original and
approximate networks. We discuss the relationship between the two methods for
selecting parameters, shedding new light on IBP and its generalizations. We
also discuss the application of our new method to approximating inference
problems which are exponential in constrained treewidth, including MAP and
nonmyopic value of information.
