reinforcement learning pst self state previous state action self select_action(pstself state,reward self env do(actionself acc_rewards +reward self [(pst,action,self state)self get((pstaction,self state), )+ if (pst,actionin self visitsself visits[(pst,action)+ self [(pst,action)+(reward-self [(pst,action)])/self visits[(pst,action)self res_states[(pst,action)add(self stateelseself visits[(pst,action) self [(pst,action)reward self res_states[(pst,action){self stateself visits_list append((pst,action)st,act pst,action #initial state-action pair for avi for update in range(self updates_per_step)self [(st,act)self [(st,act)]+self discount*sum(self [st,act,rst]/self visits[st,act]max(self get((rst,nact),self qinitfor nact in self actionsfor rst in self res_states[(st,act)])st,act random choice(self visits_listrlmodellearner py -(continued def select_action(selfstate)"""returns an action to carry out for the current agent given the stateand the -function ""if flip(self explore)return random choice(self actionselsereturn argmaxe((next_actself get((statenext_act),self qinit)for next_act in self actionsrlmodellearner py -(continued from rlqtest import senv simple game environment mbl model_based_reinforcement_learner(senv, ,updates_per_step= plot_rl(mbl ,steps_explore= ,steps_exploit= ,label="model-based( )"mbl model_based_reinforcement_learner(senv, ,updates_per_step= plot_rl(mbl ,steps_explore= ,steps_exploit= ,label="model-based( )"exercise if there was only one update per stepthe algorithm can be made simpler and use less space explain how does it make it more efficientis it worthwhile having more than one update per step for the games implemented hereversion june