storing their solutionsthose solutions being the values that associated with each possible action at each state ideallyusing memory-based data structurewellof course need to store those values and associate them with states somehowrightthe next time the same subproblem occursthe next time pac-man is in given state that have set of values for instead of recomputing its solutionone simply looks up the previously computed solutionthe value already have from the exploration stage thereby saving computation time at the expense of modest expenditure in storage spacethat' exactly what we just did with reinforcement learning we have complicated exploration phase that finds the optimal rewards associated with each action for given state once we have that table of the right action to take for given statewe can very quickly use that to make our pac-man move in an optimal manner in whole new maze that he hasn' seen before soreinforcement learning is also form of dynamic programming wowto recapyou can make an intelligent pac-man agent by just having it semi-randomly explore different choices of movement given different conditionswhere those choices are actions and those conditions are states we keep track of the reward or penalty associated with each action or state as we goand we can actually discountgoing back multiple steps if you want to make it even better then we store those values that we end up associating with each stateand we can use that to inform its future choices so we can go into whole new mazeand have really smart pac-man that can avoid the ghosts and eat them up pretty effectivelyall on its own it' pretty simple conceptvery powerful though you can also say that you understand bunch of fancy terms because it' all called the same thing -learningreinforcement learningmarkov decision processesdynamic programmingall tied up in the same concept don' knowi think it' pretty cool that you can actually make sort of an artificially intelligent pac-man through such simple techniqueand it really does workif you want to go look at it in more detailfollowing are few examples you can look at that have one actual source code you can look atand potentially play withpython markov decision process toolboxiuuq qznequppmcpysfbeuifepdtpshfombuftubqjneqiunm