Q-learning with a state-action-state reward structure and a Q-matrix with states as rows and actions as columns
There is a problem in your definition of the problem.

Q(s,a) is the expected utility of taking action a in state s and following the optimal policy afterwards.

The expected utility of a certain state (based on your definition) is different after taking 1, 2 or 9 steps. That means that the reward of being in state s_0 and taking action a_0 is different in step 0 from what you get in step 9

To adequate model the problem, you should reframe it and consider the state to be both the 'position'+'step'. You will now have 90 states (10pos*9steps).