Converging to a wrong optimal policy if the agent is given more choices
Reinforcement Learning is really fun because the agent will find any bug in your implementation and will exploit it.
&gt;&gt;&gt; take_left(0)
0
&gt;&gt;&gt; take_left(1)
-4

The agent figured out your bug with negative values and exploits negative indexing to get to the target faster.