Entropy 2025, 27, 62 9 of 33
F[q(s), o] =DKL[q(s)∥p(s)]| {z }
Complexity
−Eq(s)[ln p(o|s)]
| {z }
Accuracy
(10)
Since the VFE can be calculated (Equation (5)), it can be used as a target for minimisation.
This allows us to choose the approximate posterior q(s) that is associated with the smallest
VFE; since the surprise does not depend on q(s), minimising the VFE this way must
necessarily reduce the divergence between the approximate and exact posterior. We now,
as is usually performed in variational inference, introduce a mean-field approximation [35]
so that the joint approximate posterior q(st) factorises across time steps t ∈ T and hidden
state factors f ∈ F:
q(s) =q(s1, s2, . . . ,sT) =
T
∏
t=1
q(st) (11)
q(st) =q(s1
t , s2
t , . . . ,sF
t ) =
F
∏
f =1
q(sf
t ) (12)
The factorisation into hidden state factors allows us to calculate the VFE separately for each
factor f and sum them to obtain the total VFE. The factorisation in time allows us to calculate
the time-specific VFE as in Equation (6) using the predictive posterior ln p(st | st−1, ut−1)
from the last time step as the prior:
Ft = Eq(st)[ln q(st) − ln p(ot | st) − ln p(st | st−1, ut−1)] (13)
which is the value we intend to minimise. Various methods exist for minimising the
VFE; the one used in pymdp, from which we drew much of our inspiration, is Coordinate
Ascent Variational Inference (CAVI) [35], where the fixed points of the VFE are solved for
with coordinate descent (also known as fixed-point iteration (FPI) [ 23]). This is also the
algorithm currently available in ActiveInference.jl. We correspondingly use a coordinate
descent update to find the factorised approximate posterior q(sf
t ) that minimises the time-
dependent VFE Ft, and therefore optimises for the time-specific variational posterior q(st).
To obtain the coordinate descent update, we start by taking the derivative ofFt with respect
to q(sf
t ) and setting the derivative to zero [23]:
∂Ft
∂q(sf
t )
= ln q(sf
t ) +1 −Eqi\f [ln P(ot | st)] −ln

EP(sf
t−1,uf
t−1)
h
P(sf
t | sf
t−1, uf
t−1)
i
= 0 (14)
Solving for q(sf
t ) yields
ln q(sf
t ) =Eqi\f [ln P(ot | st)] + ln

EP(sf
t−1,uf
t−1)
h
P(sf
t | sf
t−1, uf
t−1)
i
− 1 (15)
which leads us to the coordinate descent update equation:
q∗(sf
t ) =σ

Eqi\f [ln P(ot | st)] + ln

EP(sf
t−1,uf
t−1)
h
P(sf
t | sf
t−1, uf
t−1)
i
(16)
where Eqi\f denotes the expectation over q(s) for factor i, where the posterior over states
in the other factors f are kept constant. By iteratively solving (16), the FPI scheme will
eventually find a local optimum and converge to a solution for the variational posterior.
By default, ActiveInference uses 10 iterations or stops when ∂Ft < 0.001. This posterior
then comprises the AIF agent’s belief about the state of the environment, and therefore,
its perception.