Title: Scalable Generalized Dynamic Topic Models,
Abstract: Dynamic topic models (DTMs) model the evolution of prevalent themes in
literature, online media, and other forms of text over time. DTMs assume that
word co-occurrence statistics change continuously and therefore impose
continuous stochastic process priors on their model parameters. These dynamical
priors make inference much harder than in regular topic models, and also limit
scalability. In this paper, we present several new results around DTMs. First,
we extend the class of tractable priors from Wiener processes to the generic
class of Gaussian processes (GPs). This allows us to explore topics that
develop smoothly over time, that have a long-term memory or are temporally
concentrated (for event detection). Second, we show how to perform scalable
approximate inference in these models based on ideas around stochastic
variational inference and sparse Gaussian processes. This way we can train a
rich family of DTMs to massive data. Our experiments on several large-scale
datasets show that our generalized model allows us to find interesting patterns
that were not accessible by previous approaches.