planning with uncertainty markov decision processes we will represent markov decision process (mdpdirectlyrather than using the recursive conditioning or variable elimination codeas we did for decision networks mdpproblem py -representations for markov decision processes from utilities import argmaxd import random import matplotlib pyplot as plt from matplotlib widgets import buttoncheckbuttons class mdp(object)""" markov decision process must defineself states the set (or listof states self actions the set (or listof actions self discount real-valued discount "" def __init__(selfstatesactionsdiscountinit= )self states states self actions actions self discount discount self initv self { :init for in self statesself initq self { {ainit for in self actionsfor in self states def (self, , )"""transition probability function returns dictionary of { : such that ( , )= other probabilities are zero ""raise notimplementederror(" "abstract method def (self, , )"""reward function ( ,areturns the expected reward for doing in state ""raise notimplementederror(" "abstract method two state partying example (example in poole and mackworth [ ])mdpexamples py -mdp examples from mdpproblem import mdpgridmdp class party(mdp)"""simple -state -action partying mdp example""def __init__(selfdiscount= )states {'healthy','sick'actions {'relax''party'mdp __init__(selfstatesactionsdiscountversion june