def policy_evaluation(pi, U, mdp, k=<NUM_LIT:20>):
R, T, gamma = mdp.R, mdp.T, mdp.gamma<EOL>for i in range(k):<EOL><INDENT>for s in mdp.states:<EOL><INDENT>U[s] = R(s) + gamma * sum([p * U[s1] for (p, s1) in T(s, pi[s])])<EOL><DEDENT><DEDENT>return U<EOL>
Return an updated utility mapping U from each state in the MDP to its
    utility, using an approximation (modified policy iteration).