Entropy 2025, 27, 62 23 of 33
 
aif_agent = init_aif (
A, B, C = C, D = D, E = E, pA = pA, settings = settings , parameters = parameters
)
 
4.2. Simulating Behaviour
Since the environment and agent were set up, we could proceed to simulate the
behaviour of the agent in the environment. We created afor loop, where the agent received
an observation, made inferences about the environment, updated A, inferred policies
and sampled actions:
 
# Run 1000 time steps
T = 1000
# Sample initial observation
obs = reset_TMaze !( Env )
# For loop over every time step
for t = 1:T
# Infer states based on the current observation
infer_states !( aif_agent , obs )
# Infer policies and calculate expected free energy
infer_policies !( aif_agent )
# Updating A. This is the learning part ,
# that includes the counting of the pA Dirichlet .
update_parameters !( aif_agent )
# Sample an action based on the inferred policies
chosen_action = sample_action !( aif_agent )
# Feed the action into the environment and get new observation .
obs = step_TMaze !( Env , chosen_action )
end
 
The agent here started by moving to the cue location, and then proceeded to move to the
left arm repeatedly. The main objects of learning here were the reward condition state and
the A parameters for rewards under the two reward conditions. After observing the cue,
the agent updated its belief (correctly) to be certain of being in the left reward condition
(Figure 4). Over time, the agent learned the correct probability of receiving rewards in the
left arm (0.94 versus a correct 0.95). It did not learn the probabilities for the right arm; this
was because it never moved to that location, having already learnt that the left arm was
more likely to produce rewards (Figure 5). This would be less likely to be the case with
lower γ and α values, as well as a more entropic C.
Figure 4. State inference for the reward condition. The agent’s beliefs about the reward condition
changed from agnostic (left) to certain that it was the left reward condition (right) after observing
the cue.