Title: Communication-Free Parallel Supervised Topic Models,
Abstract: Embarrassingly (communication-free) parallel Markov chain Monte Carlo (MCMC)
methods are commonly used in learning graphical models. However, MCMC cannot be
directly applied in learning topic models because of the quasi-ergodicity
problem caused by multimodal distribution of topics. In this paper, we develop
an embarrassingly parallel MCMC algorithm for sLDA. Our algorithm works by
switching the order of sampled topics combination and labeling variable
prediction in sLDA, in which it overcomes the quasi-ergodicity problem because
high-dimension topics that follow a multimodal distribution are projected into
one-dimension document labels that follow a unimodal distribution. Our
empirical experiments confirm that the out-of-sample prediction performance
using our embarrassingly parallel algorithm is comparable to non-parallel sLDA
while the computation time is significantly reduced.