A Unifying Probabilistic Perspective for Spectral Dimensionality
  Reduction: Insights and New Models
  We introduce a new perspective on spectral dimensionality reduction which
views these methods as Gaussian Markov random fields (GRFs). Our unifying
perspective is based on the maximum entropy principle which is in turn inspired
by maximum variance unfolding. The resulting model, which we call maximum
entropy unfolding (MEU) is a nonlinear generalization of principal component
analysis. We relate the model to Laplacian eigenmaps and isomap. We show that
parameter fitting in the locally linear embedding (LLE) is approximate maximum
likelihood MEU. We introduce a variant of LLE that performs maximum likelihood
exactly: Acyclic LLE (ALLE). We show that MEU and ALLE are competitive with the
leading spectral approaches on a robot navigation visualization and a human
motion capture data set. Finally the maximum likelihood perspective allows us
to introduce a new approach to dimensionality reduction based on L1
regularization of the Gaussian random field via the graphical lasso.
