Entropy 2025, 27, 62 19 of 33
 
using ActiveInference . Environments
# setting the probability of reward to 0.95
Env = TMazeEnv (0 .9 5)
initialize_gp ( Env )
 
We then proceeded to set upA, B, C, D and E. For this we used thecreate_matrix_templates
helper function to set up the correct structure and then populate it. To start, we defined
what went into the helper function. It took five arguments: the numbers of states and ob-
servations in each factor and modality; the policy length; the number of controls; and lastly,
what to initially populate them with. The specific states and observations are made clear
below when discussing populating the parameters.
The first three arguments should be specified as vectors of integers, containing the
numbers of states and observations for each factor or modality. In our case, we had two
factors: a location factor and a reward condition factor, which had four and two states,
respectively. There were three modalities: one location modality with four observations,
one reward modality with three observations and one cue modality with two observations.
Finally, there were four possible actions for controlling the location factor, and only one
possible action for the reward state factor. The policy length was specified as an integer,
in our case 2, and we populated the template with zeros.
 
A, B, C, D, E = create_matrix_templates ([4, 2], [4, 3, 2], [4, 1], 2, " zeros ")
 
We started by defining A, or the observation model. In this example, since we allowed
for A learning, A was not used directly, but we still defined it in order to construct the
Dirichlet prior over it (and it could be used directly if A learning was not required). Of the
three observation modalities, the first was the location observation. Here, there were
four possible observations, mapped to the four location states and two reward condition
states. This resulted in an A that was four location observations by four location states by
two reward conditions, i.e., a 4 × 4 × 2 tensor. We let the agent correctly assume perfect
observations of the location by specifying an identity matrix forA in each reward condition:
 
# Use identity matrices between locations and observations .
# For reward condition right
A[1][:,:,1] = [ 1 .0 0 .0 0 .0 0 .0
0 .0 1 .0 0 .0 0 .0
0 .0 0 .0 1 .0 0 .0
0 .0 0 .0 0 .0 1 .0 ]
# For reward condition left
A[1][:,:,2] = [ 1 .0 0 .0 0 .0 0 .0
0 .0 1 .0 0 .0 0 .0
0 .0 0 .0 1 .0 0 .0
0 .0 0 .0 0 .0 1 .0 ]
 
The second modality was the reward modality, which mapped the observations “no re-
ward”, “reward” and “loss” onto the location states and reward conditions. For the second
modality, we therefore had a tensor that was three reward observations by four location
states by two reward conditions, i.e., a 3 × 4 × 2 tensor. When the agent was at the centre
and cue locations, we let the modality accurately expect the observation of “no reward”
with certainty. For the two arm locations, we let the agent be agnostic regarding whether
they provided rewards or losses. This was different from the true reward probabilities
(see Figure 3), which the agent needed to learn over time. This was the case for both
reward conditions.