given particular stateit' easy for us to make decisionwe simply look at each possible action and its rewardthen take the action that corresponds with the highest expected reward reformulating the earlier policy more formallywe havepolicystate -argmax_{actionq(stateactionthis satisfies the requirements of decision-making functiongiven state in the gameit decides on an action howeverthis solution depends on knowing (stateactionfor every state and action to estimate (stateaction)consider the following given many observations of an agent' statesactionsand rewardsone can obtain an estimate of the reward for every state and action by taking running average space invaders is game with delayed rewardsthe player is rewarded when the alien is blown up and not when the player shoots howeverthe player taking an action by shooting is the true impetus for the reward somehowthe -function must assign (state shoota positive reward these two insights are codified in the following equationsq(stateaction( learning_rateq(stateactionlearning_rate q_target q_target reward discount_factor max_{action' (state'action'