text: # For action " Move to Cue Location " 
  B[1][:,:,4] = [ 0 .0 
  0 .0 
  0 .0 
  1 .0  0 .0 
  0 .0 
  0 .0 
  1 .0  0 .0 
  0 .0 
  0 .0 
  1 .0  0 .0 
  0 .0 
  0 .0 
  1 .0 ] 
  (cid:6)  (cid:5) 
  
  The reward condition factor in**B**here was rather simple, as we let the agent correctly 
  assume that the reward condition never changed (although one could have enabled**B**
  learning to let the agent learn the transition probabilities of the environment). There were 
  two possible states for these state factors, so we implemented this as a 2 × 2 identity matrix: 
  
  (cid:7)  (cid:4) 
  
  # Identity matrix representing agent belief that reward condition does not change 
  B[2][:,:,1] = [ 1 .0 
  0 .0  0 .0 
  1 .0 ] 
  (cid:6)  (cid:5) 
  
  Then, we built**C**, the prior preference over observations.**C**encoded preferences for each 
  observation in each modality: the location, reward and cue. The relative value of the entry 
  corresponding to a specific observation defined the preference; here, we set all values with 
  neutral preferences as 0, and used positive values for preferences and negative values for 
  aversions. The only observations over which the agent had preferences were in the reward 
  modality, where the rewards (index 2) were preferred and losses (index 3) were disliked. 
  The strength of the preference determined the explore–exploit balance of the agent; here, 
  we use 3 as the value: 
  (cid:7)  (cid:4) 
  
  # Preference over locations modality 
  C[1] = [0 .0 , 0 .0 , 0 .0 , 0 .0 ] # No preference 