somdpsmarkov decision processesare fancy way of describing our exploration algorithm that we just described for reinforcement learning the notation is even similarstates are still described as sand sis the next state that we encounter we have state transition functions that are defined as pa for given state of and swe have our valueswhich are basically represented as reward functionan ra value for given and ssomoving from one state to another has given reward associated with itand moving from one state to another is defined by state transition functionstates are still described as and 'state transition functions are described as pa( , 'our values are described as reward function ra( , 'so againdescribing what we just didonly mathematical notationand fancier sounding wordmarkov decision processes andif you want to sound even smarteryou can also call markov decision process by another namea discrete time stochastic control process that sounds intelligentbut the concept itself is the same thing that we just described dynamic programming soeven more fancy wordsdynamic programming can be used to describe what we just did as well wowthat sounds like artificial intelligencecomputers programming themselvesterminator skynet stuff but noit' just what we just did if you look up the definition of dynamic programmingit is method for solving complex problem by breaking it down into collection of simpler subproblemssolving each of those subproblems just onceand storing their solutions ideallyusing memory-based data structure the next time the same subproblem occursinstead of recomputing its solutionone simply looks up the previously computed solution thereby saving computation time at the expense of (hopefullymodest expenditure in storage spacea method for solving complex problemsame as creating an intelligent pacmanthat' pretty complicated end result by breaking it down into collection of simpler subproblemssofor examplewhat is the optimal action to take for given state that pac-man might be in there are many different states pac-man could find himself inbut each one of those states represents simpler subproblemwhere there' limited set of choices can makeand there' one right answer for the best move to make