without any human intervention was developed by the researchers at deepmindwho also trained their agent to play variety of atari games deepmind' original deep -learning (dqnpaper recognized two important issues correlated statestake the state of our game at time which we will call say we update ( )according to the rules we derived previously nowtake the state at time which we call and update ( according to the same rules note that the game' state at time is very similar to its state at time in space invadersfor examplethe aliens may have moved by one pixel each said more succinctlys and are very similar likewisewe also expect ( and ( to be very similarso updating one affects the other this leads to fluctuating valuesas an update to ( may in fact counter the update to ( more formallys and are correlated since the -function is deterministicq( is correlated with ( -function instabilityrecall that the function is both the model we train and the source of our labels say that our labels are randomly-selected values that truly represent distributionl every time we update qwe change lmeaning that our model is trying to learn moving target this is an issueas the models we use assume fixed distribution to combat correlated states and an unstable -function one could keep list of states called replay buffer each time stepyou add the game state that you observe to this replay buffer you