Entropy 2025, 27, 62 24 of 33
Figure 5. A learning for the actual reward condition (reward condition left). The agent correctly
learned the probability of receiving rewards in the rewarding arm. It did not learn the probabilities of
the non-rewarding arm since it did not explore that option. The color grading signifies the likelihood
of an observation being generated by a specific state. The more saturated the color, the higher
the likelihood.
4.3. Fitting the Model to the Data
Simulations are useful for a variety of purposes, like exploring the consequences of
different priors and parameters and establishing the face validity of hypothetical mech-
anisms underlying behavioural phenomena. However, we often want to use models to
make inferences about specific observed phenomena, like the differences in behaviour
between various populations, as in computational psychiatry [14]. One standard method
here is model fitting, where we estimate the parameter values (e.g., prior beliefs) of an AIF
model that are the most likely given some observed behaviour of a participant. This is often
performed with approximate Bayesian methods. In the cognitive and behavioural sciences,
the predominant method is Markov Chain Monte Carlo (MCMC) methods [34], which are
slower but in the limit can estimate parameter posteriors without making assumptions
about their functional form. An alternative, which is more often used in other fields and
also available in ActiveInference is variational methods, which are faster but require mak-
ing assumptions about the functional form of the posterior. In general, MCMC methods
are favourable when making parameter inferences (i.e., comparing parameters of the same
model fitted to different data, like two groups of subjects). When performing a Bayesian
model comparison (i.e., comparing different models fitted to the same data), the different
approaches rely on different approximations of the model evidence, with the variational