Operations for Learning with Graphical Models
  This paper is a multidisciplinary review of empirical, statistical learning
from a graphical model perspective. Well-known examples of graphical models
include Bayesian networks, directed graphs representing a Markov chain, and
undirected networks representing a Markov field. These graphical models are
extended to model data analysis and empirical learning using the notation of
plates. Graphical operations for simplifying and manipulating a problem are
provided including decomposition, differentiation, and the manipulation of
probability models from the exponential family. Two standard algorithm schemas
for learning are reviewed in a graphical framework: Gibbs sampling and the
expectation maximization algorithm. Using these operations and schemas, some
popular algorithms can be synthesized from their graphical specification. This
includes versions of linear regression, techniques for feed-forward networks,
and learning Gaussian and discrete Bayesian networks from data. The paper
concludes by sketching some implications for data analysis and summarizing how
some popular algorithms fall within the framework presented. The main original
contributions here are the decomposition techniques and the demonstration that
graphical models provide a framework for understanding and developing complex
learning algorithms.
