Entropy 2025, 27, 62 25 of 33
free energy having some claims to being a better approximation than the information
criteria classically used with MCMC methods (although see other approximations, like the
Pareto-Smoothed Importance Sampling [59] or Thermodynamic Integration methods [60];
see [35] for a further review). Note that independently of which of these approaches one
might take, the process involves inverting a generative model of the mental processes
underlying the behaviour of a given subject, a generative model which itself is an inversion
of the subject’s generative model of the environment. We can call the generative model that
the agent has of its environment the subjective generative model, and the model we have of
the agent the objective generative model, in what has been called a meta-Bayesian approach
or “observing the observer” [1,61].
Here, we demonstrated model fitting by fitting the POMDP model to the synthetic
behaviour that it generated; this is called a parameter recovery study since we can then
compare the estimated parameters to the generative values used for creating the simulated
data [62,63]. Here, we used the simulation method shown in the previous section to
produce a synthetic dataset with known parameter values for each agent (in practice, these
are often participants in an experiment), here with a focus on estimating the α parameter.
We then used MCMC methods to estimate the parameters for each agent and compared
the estimated values with the correct values. Here, we simulated two groups of five
synthetic subjects agents with different α values (the parameters for the first group were
sampled from a Gaussian distribution with mean = 8 and SD = 2, and the second group
with with mean = 24 and SD = 2). Each agent interacted with the T-maze environment for
300 time steps. We produced the following data frame, containing the data of each of the
agents: their observations, actions and an identifier, a format suitable for cognitive and
behavioural modelling.
3000×5 DataFrame
Row Location Reward Cue Action_Location Action_Reward SubjectID
Int64 Int64 Int64 Int64 Int64 Int64
1 1 1 1 4 1 1
2 4 1 2 3 1 1
3 3 3 2 2 1 1
. . . . . . .
. . . . . . .
3000 2 2 2 2 1 10
We usedActionModels to fit the AIF model created above to each of the agents in the dataset.
We began by initialising an ActionModels agent:
 
using ActionModels
# Initialize ActionModels Agent with the action model and created active inference agent
agent = init_agent (
action_model = action_pomdp !, # Action model function
substruct = aif , # Active inference agent as a substruct
)
 
We then set the prior for the parameter we wanted to estimate: the α action precision. As
an example, we chose a wide, weakly informative prior: a Gaussian distribution with mean
5 and standard deviation 5, truncated at 0 and 20: