Entropy 2025, 27, 62 18 of 33
In the following, we proceed to describe the generative model used by the agent,
which here was identically structured to the generative process, the actual environmental
process generating the data. In the maze task, the environmental states and concurrent
observations were multidimensional, while the actions were unidimensional. Here, we refer
to the different dimensions of the environment as state factorsand the different dimensions
of the observations as observation modalities. There were two state factors, the location of the
agent and the reward condition:
State Factor 1 (Location):
1. Centre location;
2. Right arm;
3. Left arm;
4. Cue location.
State Factor 2 (Reward Condition):
1. Reward condition right;
2. Reward condition left.
The first factor described which of the four locations in the maze the agent was
inhabiting—the centre, cue location, and left and right arms—and was controlled by the
action. The second factor was the reward condition factor, and had two possible states: the
reward location being on the right or on the left. The agent’s actions directly controlled the
first factor, while it had to infer the second based on the cue in order to complete the task.
There were three observation modalities: one for observing the agent’s own location, one
for observing rewards and one for observing the cue:
Modality 1 (Location):
1. Centre location;
2. Right arm;
3. Left arm;
4. Cue location.
Modality 2 (Reward):
1. No reward;
2. Reward;
3. Loss.
Modality 3 (Cue):
1. Cue right;
2. Cue left.
The first observation modality had an observation for each of the positions in the maze,
and generally reflected the agent’s actual position perfectly. The second modality was the
agent’s observation of a reward or loss, which depended on the reward condition and the
position. The “no reward” observation was received whenever the agent was not occupying
either of the arm locations, where it did not observe any reward or loss. The third modality
was the observation of the cue, which depended on the reward modality so that the agent
could use it to infer the correct reward condition. The action was one-dimensional with four
options and was used to move between the different locations:
Actions:
1. Move to centre location;
2. Move to right arm;
3. Move to left arm;
4. Move to cue location.
We now provide an example of using the package with a T-maze environment. First,
we installed and loaded the package:
 
using Pkg
Pkg . add ( ActiveInference )
using ActiveInference
 
We first set up the environment or generative process: the T-maze. The T-maze is included
as a pre-made environment in ActiveInference, so we simply loaded it. We set the reward
probability for the reward condition to 95 percent: