Entropy 2025, 27, 62 7 of 33
Figure 1. Depiction of a POMDP generative model. This encodes the agent’s expectations about how
the state s of the environment changes over timet, and how it generates observationo at each time step.
A, also called the observation model, describes how environmental states give rise to observations.B,
also called the transition model, describes how environmental states change over time, depending on
action u (called policy π when structured into sequences).C is the preference prior, which encodes
the agent’s preferences for observations. This shapes the expected free energyG associated with each
policy , which is used for policy selection.D encodes the agent’s prior belief over environmental states
before making any observations, andE is the prior over policies that determines the agent’s preferences
for policies in the absence of other motivation.
2.2. Perception in Active Inference
In AIF, perception is conceptualised as the result of variational (i.e., approximate)
Bayesian inference, performed by minimising the VFE to optimise parameters of poste-
rior beliefs about the environment. In exact Bayesian inference, we use a parametrised
generative model m to make an optimal inference about state s of the environment based
on observation o. This is performed by combining a prior belief over states p(s|m); a like-
lihood model p(o|s, m); and the model evidence p(o|m), a normalisation term encoding
the likelihood of receiving the given observations across all possible environmental states,
as follows [1]:
p(s|o, m) = p(o|s, m)p(s|m)
p(o|m) (1)
The posterior distribution over states given observationsp(s|o, m) here represent the agent’s
beliefs about the environment. Forming beliefs in this way is thought to be the process that
enables conscious, as well as unconscious, perception. The product of the likelihood model
and prior is also called the joint likelihood p(o, s|m), which fully defines the generative
model, and which we use henceforth. In the following, for notational simplicity, we also
omit denoting the dependency on the generative model m.
Calculating the model evidence p(o) is often intractable, making exact Bayesian infer-
ence unfeasible. The way to circumvent this in AIF is to use a variational approximation
to Bayesian inference [23,33,50,51]. This works by transforming the inference into an opti-
misation problem, specifically the minimisation of the VFE. First, an arbitrary probability
distribution over environmental states q(s), an approximate posterior that is used to ap-
proximate the exact posterior, is introduced. We then introduce the Kullback–Leibler (KL)