Entropy 2025, 27, 62 17 of 33
parameter recovery and predictive checks. In this section, we outline how to use
ActiveInference for simulation and model fitting in conjunction withActionModels. In the
following section, we show how to achieve this on a concrete worked example.
4. Usage Example
In this section, we demonstrate a full usage example of how to create an AIF agent, sim-
ulate behaviour in a classic T-maze environment and fit the AIF agent to a simulated exam-
ple dataset. We provide the necessary code to run this simulation. All code required to repro-
duce the example simulation can be found in an open source OSF repository osf.io/j3k5q/.
This example was performed with the current version of ActiveInference.jl (0.1.1); the
newest version can be found at github.com/ilabcode/ActiveInference.jl.
4.1. Setting Up Environment and Agent
A T-maze is a simple task commonly employed in the behavioural sciences, as well
as in the AIF literature [ 14,54–57]. It is a minimal type of task that requires balancing
exploration and exploitation, or epistemic and pragmatic value, respectively. It is also
suitably represents in a discrete state space. Together, this makes it easily compatible with
a POMDP-based AIF approach.
The structure of the T-maze is, as the name suggests, a T-shaped maze, consisting
of a centre location, a cue location (bottom of the T), and reward and loss locations (one
in each arm of the T) (Figure 2). On every trial, the agent can move to one of the two
arms of the T to receive a reward; one, called the reward location, will yield rewards with
a higher probability than the other side. At the cue location, which the agent can move
to, the agent receives a cue that indicates which of the locations is the reward location.
Generally, the cue may be more or less informative; in this example, it always accurately
reflected the reward conditions state (reward in the right or left arm). The reward location
only provides a reward probabilistically. This means the agent can either take a chance
and go directly to one of the two upper arms, or spend its first move seeking information
about where the reward is before moving to the reward location. Since the clue location
is not preferred, the second option comes with a cost in terms of pragmatic value, which
has to be outweighed by the epistemic value in resolving uncertainty about the reward
location state. Note that for the agent to realise that this uncertainty reduction will aid it in
its subsequent choice of arm, it would have to be able to anticipate the effect of its actions
on its own future beliefs, a process called “sophisticated inference” [58].
Figure 2. A depiction of the T-maze. (A) The full layout of the T-maze task, with the centre location,
the cue location and the two reward conditions. (B) A three-step example of a T-maze trial. The agent
(in this case, a mouse) starts at the centre location. In order to reduce the uncertainty regarding which
arm the reward is located in, the agent moves to the cue location. The cue location reveals the right
arm to be the reward location, and in the subsequent time step, it goes to the right arm and observes
the reward with some probability.