Entropy 2025, 27, 62 2 of 33
minimisation [9]. Choosing actions that minimise the expected free energy (EFE) of their
consequences provides a natural balance between exploratory and exploitative behaviour;
generalises descriptive approaches to behavioural modelling, like reinforcement learning
and expected utility maximisation; and provides a singular approach to adaptive behaviour
that can be used across different environments. AIF was argued to be applicable to any self-
organising system that actively maintains a stable boundary that defines its integrity [10],
a broad category that includes cells and plants [ 11], as well as humans [ 2] and even
collectives [12]. Owing to its generality, AIF has seen a rise in popularity across multiple
fields. It is used for theoretical simulations of the mechanisms underlying various types of
behaviour [2], computational phenotyping in computational psychiatry [13,14], and agent-
based simulations of population dynamics [15], as well as in engineering and robotics [16].
In AIF, perception and concurrent action are based on performing a variational Bayesian
inversion of a generative model of the environment (i.e., a model of how the environment
changes and brings about sensory observations). This belief updating includes inferring
(hidden) states of the environment, learning parameters of the generative model and
learning the structure of the generative model. Since the requisite inference schemes come
pre-specified, the main task in AIF modelling becomes specifying an appropriate generative
model. This includes specifying priors over environmental states, as well as what might
be called prior preferences, preference priorsor goal priors: immutable prior expectations that
make up an agents’ preferences by furnishing a set of predictions over future states or
observations; in fulfilling these predictions, free energy is minimised. The space of possible
generative models is vast, and they often have to be handcrafted for a given environment.
However, there are some families of generative models that can be considered “universal”
in the sense that they can be used for most environments. Currently, the most popular of
these is the discrete state-space Partially Observable Markov Decision Process (POMDP)-
based generative models. Since they are ubiquitous in the literature, we focus here on
making these types of generative models available to researchers. There are, however, other
types of universal generative models, like generalised filtering models [17] or Hierarchical
Gaussian Filtering-based models [18,19], that will be implemented in the future.
Tools for simulating POMDP-AIF models were originally developed as part of the
DEM [20] library for MATLAB [21] (part of the larger SPM library [ 22]). Since then, a
modal and flexible software package pymdp [23] was created for Python [24], as well as a
performance-oriented package cpp-AIF [25] for C++ [26] that can be used across platforms.
Finally, the factor graph library RxInfer [27] for Julia [28] has also been used to implement
some AIF models on an efficient factor graph back-end [ 29–31]. The important tools
that these packages provide make AIF available for researchers to perform simulation
studies and for use in engineering contexts. They do not, however, usually allow for
fitting models to empirically observed data, which is a fundamental method used in
cognitive modelling [32], often in the context of computational psychiatry [13], to infer the
mechanisms underlying variations in behaviour or to investigate the differences between
(for example, clinical) populations. Smith and colleagues [33] provided a guide for manually
doing variational Bayesian parameter estimation based on empirical data, but only in
MATLAB and restricted to a particular class of variational parameter estimation methods
(variational Laplace), instead of the sampling-based methods that currently predominate in
the field of cognitive modelling [34,35].