Title: Ising Models with Latent Conditional Gaussian Variables,
Abstract: Ising models describe the joint probability distribution of a vector of
binary feature variables. Typically, not all the variables interact with each
other and one is interested in learning the presumably sparse network structure
of the interacting variables. However, in the presence of latent variables, the
conventional method of learning a sparse model might fail. This is because the
latent variables induce indirect interactions of the observed variables. In the
case of only a few latent conditional Gaussian variables these spurious
interactions contribute an additional low-rank component to the interaction
parameters of the observed Ising model. Therefore, we propose to learn a sparse
+ low-rank decomposition of the parameters of an Ising model using a convex
regularized likelihood problem. We show that the same problem can be obtained
as the dual of a maximum-entropy problem with a new type of relaxation, where
the sample means collectively need to match the expected values only up to a
given tolerance. The solution to the convex optimization problem has
consistency properties in the high-dimensional setting, where the number of
observed binary variables and the number of latent conditional Gaussian
variables are allowed to grow with the number of training samples.