Optimization motion planning by using Bellman equation
From my understanding that agent is a ball, environment is the plane, action is rolling ball without sliding and the achieved goal is motion planning from 1 point to another point.


That's already a good start. Still, I would recommend to be a bit more concrete to clear everything up. The action is not only the rolling ball, but has to be some kind of instructive control, e.g. move the ball 1cm forward, steer 10° clockwise, accellerate by a - just to name a few possibilities. Given this Action set $A$ with $a\in A$. $P_{ss\prime}^a$ models the state transition probabilities - if you are in a deterministic environment, it is constant 1 for one state and 0 for all others. $R$ is the reward, and here is where it becomes tricky. You want to find the best trajectory to the goal, thus you want to penalize if the agent takes unneccessary actions, leading to the agent reaching the goal as fast as possible. I would recommend a constant -1 for that purpose, if you end the episode when the ball reaches the goal. If this is not the case, it has to be more sophisticated, e.g. the negative distance of the ball towards the goal. This should be everything you need to get your hands dirty.