Entropy 2025, 27, 62 22 of 33
 
# For the location state factor
D[1] = [1 .0 , 0 .0 , 0 .0 , 0 .0 ] # Certain beleif of being at center location
# For the reward condition state factor
D[2] = [0 .5 , 0 .5 ] # Agnostic prior over reward condition state
 
Finally, we establishedE, the prior over policies. This was a vector over all possible policies;
with four possible actions and a policy length of 2, there were 42 = 16 policies in total.
Here, we gave the agent an agnostic prior over policies:
 
# Setting uniform E vector to not prefer any policies by equally preferring all
E .= 1 .0 / length (E)
 
Thus, the five matrices were defined (we note that the C, D and E matrices were set to
agnostic by default if not defined by the user). All that was left was to set the Dirichlet
prior for A learning, and set the parameters and settings. In the following, we set the the
Dirichlet prior for A to a scaled copy of the original A using a weak scaling parameter
of 2.0:
 
# Use a weak scaling parameter
scale_concentration_parameter = 2 .0
# Make a scaled copy of A
pA = deepcopy (A) * scale_concentration_parameter
 
We then set the various hyper-parameters for the agent’s inference algorithm. Here, we
used standard default values: an A learning rate of 1, and γ and α values of 16:
 
parameters = Dict (
" lr_pA " => 1 .0 ,
" alpha " => 16 .0 ,
" gamma " => 16 .0 ,
)
 
Finally, we defined the settings of the agent (other characteristics of the agent that were not
estimable parameters). Here, we set the policy length to 2, which let the agent use expected
information gain for both the state and parameter info gain for its actions, and specify that
it was specifically the second modality of A that was to be learned:
 
settings = Dict (
# A policy length of two
" policy_len " => 2,
# Use parameter information gain in action selection
" use_param_info_gain " => true ,
# Use state information gain in action selection
" use_states_info_gain " => true ,
# Only do A learning for the reward modality .
" modalities_to_learn " => [2]
)
 
Having built all the necessary components of the generative model, we could then instanti-
ate an AIF agent: