Extending Factor Graphs so as to Unify Directed and Undirected Graphical
  Models
  The two most popular types of graphical model are directed models (Bayesian
networks) and undirected models (Markov random fields, or MRFs). Directed and
undirected models offer complementary properties in model construction,
expressing conditional independencies, expressing arbitrary factorizations of
joint distributions, and formulating message-passing inference algorithms. We
show that the strengths of these two representations can be combined in a
single type of graphical model called a 'factor graph'. Every Bayesian network
or MRF can be easily converted to a factor graph that expresses the same
conditional independencies, expresses the same factorization of the joint
distribution, and can be used for probabilistic inference through application
of a single, simple message-passing algorithm. In contrast to chain graphs,
where message-passing is implemented on a hypergraph, message-passing can be
directly implemented on the factor graph. We describe a modified 'Bayes-ball'
algorithm for establishing conditional independence in factor graphs, and we
show that factor graphs form a strict superset of Bayesian networks and MRFs.
In particular, we give an example of a commonly-used 'mixture of experts' model
fragment, whose independencies cannot be represented in a Bayesian network or
an MRF, but can be represented in a factor graph. We finish by giving examples
of real-world problems that are not well suited to representation in Bayesian
networks and MRFs, but are well-suited to representation in factor graphs.
