Title: Prior-aware Dual Decomposition: Document-specific Topic Inference for Spectral Topic Models,
Abstract: Spectral topic modeling algorithms operate on matrices/tensors of word
co-occurrence statistics to learn topic-specific word distributions. This
approach removes the dependence on the original documents and produces
substantial gains in efficiency and provable topic inference, but at a cost:
the model can no longer provide information about the topic composition of
individual documents. Recently Thresholded Linear Inverse (TLI) is proposed to
map the observed words of each document back to its topic composition. However,
its linear characteristics limit the inference quality without considering the
important prior information over topics. In this paper, we evaluate Simple
Probabilistic Inverse (SPI) method and novel Prior-aware Dual Decomposition
(PADD) that is capable of learning document-specific topic compositions in
parallel. Experiments show that PADD successfully leverages topic correlations
as a prior, notably outperforming TLI and learning quality topic compositions
comparable to Gibbs sampling on various data.