Title: Cooperative Hierarchical Dirichlet Processes: Superposition vs. Maximization,
Abstract: The cooperative hierarchical structure is a common and significant data
structure observed in, or adopted by, many research areas, such as: text mining
(author-paper-word) and multi-label classification (label-instance-feature).
Renowned Bayesian approaches for cooperative hierarchical structure modeling
are mostly based on topic models. However, these approaches suffer from a
serious issue in that the number of hidden topics/factors needs to be fixed in
advance and an inappropriate number may lead to overfitting or underfitting.
One elegant way to resolve this issue is Bayesian nonparametric learning, but
existing work in this area still cannot be applied to cooperative hierarchical
structure modeling.
In this paper, we propose a cooperative hierarchical Dirichlet process (CHDP)
to fill this gap. Each node in a cooperative hierarchical structure is assigned
a Dirichlet process to model its weights on the infinite hidden factors/topics.
Together with measure inheritance from hierarchical Dirichlet process, two
kinds of measure cooperation, i.e., superposition and maximization, are defined
to capture the many-to-many relationships in the cooperative hierarchical
structure. Furthermore, two constructive representations for CHDP, i.e.,
stick-breaking and international restaurant process, are designed to facilitate
the model inference. Experiments on synthetic and real-world data with
cooperative hierarchical structures demonstrate the properties and the ability
of CHDP for cooperative hierarchical structure modeling and its potential for
practical application scenarios.