text: We then proceeded to set up**A**,**B**,**C**,**D**and**E**. For this we used the create_matrix_templates 
  helper function to set up the correct structure and then populate it. To start, we defined 
  what went into the helper function. It took five arguments: the numbers of states and ob- 
  servations in each factor and modality; the policy length; the number of controls; and lastly, 
  what to initially populate them with. The specific states and observations are made clear 
  below when discussing populating the parameters. 
  
  The first three arguments should be specified as vectors of integers, containing the 
  numbers of states and observations for each factor or modality. In our case, we had two 
  factors: a location factor and a reward condition factor, which had four and two states, 
  respectively. There were three modalities: one location modality with four observations, 
  one reward modality with three observations and one cue modality with two observations. 
  Finally, there were four possible actions for controlling the location factor, and only one 
  possible action for the reward state factor. The policy length was specified as an integer, 
  in our case 2, and we populated the template with zeros. 
  (cid:7)  (cid:4) 
  
  (cid:5) 