given the stateand the -function ""if flip(self explore)return random choice(self actionselsereturn argmaxe((next_actself get((statenext_act),self qinit)for next_act in self actionsexercise implement soft-max action selection choose temperature that works well for the domain explain how you picked this temperature compare the epsilon-greedysoft-max and optimism in the face of uncertainty exercise implement sarsa hintit does not do max in do instead it needs to choose next act before it does the update testing -learning the first tests are for the -action -state rlqtest py -rl tester from rlproblem import healthy_env from rlqlearner import q_learner from rlplot import plot_rl env healthy_env(ag q_learner(env ag_opt q_learner(env qinit= label="optimisticoptimistic agent ag_exp_l q_learner(env explore= label="less explore"ag_exp_m q_learner(env explore= label="more explore"ag_disc q_learner(env qinit= label="disc "ag_va q_learner(env qinit= ,fixed_alpha=false,alpha_fun=lambda : /( + ),label="alpha= /( + )" ag max_display_level ag do( ag get the learned -values ag max_display_level ag do( ag get the learned -values plot_rl(ag,yplot="average"plot_rl(ag_opt,yplot="average"plot_rl(ag_exp_l,yplot="average"plot_rl(ag_exp_m,yplot="average"plot_rl(ag_disc,yplot="average"plot_rl(ag_va,yplot="average" from mdpexamples import mdptiny from rlproblem import env_from_mdp envt env_from_mdp(mdptiny()version june