Entropy 2025, 27, 62 6 of 33
it means that the transitions of the environment are expected to be uncertain (and therefore,
often transition to new states). In this sense, volatile and unstable environments will lead
to less certain predictions about the future.
C, also called the preference prior, is a prior preference over possible observations. It encodes
the types of observations that an agent a priori expects to encounter; since minimising ex-
pected free energy through AIF entails taking actions that make the predicted observations
come about, C also encodes the agent’s preferences. It is a single categorical probability
distribution over possible observations; if the observations are multidimensional, there
is a separate preference priorfor each observation modality. If C is imprecise (i.e., highly
entropic), its preferences are weak and it will prioritise collecting information over realising
its preferences; if it has low entropy, the agent will have stronger preferences and instead
prioritise preferred outcomes or goals.
D, also called the state prior, is the agent’s prior belief about the states of the environment. It
specifies the agent’s belief about the environmental state before receiving any observations.
There is a separate state priorover environmental states for each factor. With a more precise
A, the influence of the D quickly diminishes since the likelihood overwhelms the prior in
the Bayesian inference.
E, also called the habit prior, is the prior over policies or paths. In the AIF vernacular,
policies are allowable sequences of actions, with some specified policy length or temporal
depth. E encodes the agent’s preferences for choosing certain policies in the absence of
plans based upon expected free energy, sometimes called the agent’s “habits”. It is a single
probability distribution over each possible policy.
In addition to the five matrices, there are several hyper-parameters that are not part
of the generative model, but are part of the inference algorithm. Here, we include two of
the most common: the γ and α (inverse) temperature parameters. γ, the precision over
policies, is the inverse temperature of a softmax transformation of expected free energies
over policies, which is covered later in this section. After policies have been selected for a
given time step, they are marginalised to calculate the probabilities of taking each possible
action in the next time step. α, the action precision, is the inverse temperature of a softmax
transformation on these final action probabilities, with higher values resulting in more
stochastic action selection.
As noted, here we focus specifically on the POMDP-based generative models often
used in the AIF literature. However, the basic steps when performing AIF—perception,
action and learning—remain the same across generative models. In the remainder of this
section, we describe each of these three steps in turn.