text: **Figure 3.**Reward probabilities for the four locations. The centre (column 1) and cue (column 4) 
  locations always resulted in the “no reward” observation (row 1). The two arms (columns 3 and 
  4) resulted in either rewards (row 2) or losses (row 3), with some probability. Left: the agent’s 
  agnostic starting beliefs about reward probabilities. Right: the true reward probabilities for the 
  reward condition left arm, which the agent needed to learn over time. The amount of saturation of 
  the green color represent the likelihood of a specific observation in a give state. 
  
  The third and last modality was the cue modality, which mapped the cue observations 
  onto the location and reward condition states. This resulted in an**A**that was two cue 
  observations by four location states by two reward conditions, i.e., a 2 × 4 × 2 tensor. The 
  observations in this modality were correctly assumed by the agent to truthfully reveal the 
  current reward condition—i.e., whether the right or left arm was better—when standing at 
  the cue location. We implemented this by giving each cue observation equal probabilities 
  at all locations except the cue location, where there was a perfect correspondence between 
  the reward condition and the observation: 
  (cid:7)  (cid:4) 
  
  # Set cue observation probabilities to be equal at all 
  # locations except the cue location ( column 4). 
  # Let cue locations correspond to reward conditions . 