Title: The Consciousness Prior,
Abstract: A new prior is proposed for representation learning, which can be combined
with other priors in order to help disentangling abstract factors from each
other. It is inspired by the phenomenon of consciousness seen as the formation
of a low-dimensional combination of a few concepts constituting a conscious
thought, i.e., consciousness as awareness at a particular time instant. This
provides a powerful constraint on the representation in that such
low-dimensional thought vectors can correspond to statements about reality
which are true, highly probable, or very useful for taking decisions. The fact
that a few elements of the current state can be combined into such a predictive
or useful statement is a strong constraint and deviates considerably from the
maximum likelihood approaches to modelling data and how states unfold in the
future based on an agent's actions. Instead of making predictions in the
sensory (e.g. pixel) space, the consciousness prior allows the agent to make
predictions in the abstract space, with only a few dimensions of that space
being involved in each of these predictions. The consciousness prior also makes
it natural to map conscious states to natural language utterances or to express
classical AI knowledge in the form of facts and rules, although the conscious
states may be richer than what can be expressed easily in the form of a
sentence, a fact or a rule.