the benefit of this technique is that once you've explored the entire set of possible states that your agent can be inyou can very quickly have very good performance when you run different iterations of this soyou knowyou can basically make an intelligent pac-man by running reinforcement learning and letting it explore the values of different decisions it can make in different states and then storing that informationto very quickly make the right decision given future state that it sees in an unknown set of conditions -learning soa very specific implementation of reinforcement learning is called -learningand this formalizes what we just talked about little bit moreso againyou start with set of environmental states of the agent (is there ghost next to meis there power pill in front of methings like that )we're going to call that have set of possible actions that can take in those stateswe're going to call that set of actions in the case of pac-manthose possible actions are move updownleftor right then we have value for each state/action pair that we'll call qthat' why we call it -learning sofor each statea given set of conditions surrounding pacmana given action will have value somoving up might have given value qmoving down might have negative value if it means encountering ghostfor example sowe start off with value of for every possible state that pac-man could be in andas pac-man explores mazeas bad things happen to pac-manwe reduce the value for the state that pac-man was in at the time soif pac-man ends up getting eaten by ghostwe penalize whatever he did in that current state as good things happen to pac-manas he eats power pillor eats ghostwe'll increase the value for that actionfor the state that he was in thenwhat we can do is use those values to inform pac-man' future choicesand sort of build little intelligent agent that can perform optimallyand make perfect little pac-man from the same image of pac-man that we saw just abovewe can further define the current state of pac-man by defining that he has wall to the westempty space to the north and easta ghost to the south