assignment1_coding/vi_and_pi.py
def policy_improvement(P, nS, nA, value_from_policy, policy, gamma=0.9):
    """Given the value function from policy improve the policy.

    Parameters
    ----------
    P, nS, nA, gamma:
        defined at beginning of file
    value_from_policy: np.ndarray
        The value calculated from the policy
    policy: np.array
        The previous policy.

    Returns
    -------
    new_policy: np.ndarray[nS]
        An array of integers. Each integer is the optimal action to take
        in that state according to the environment dynamics and the
        given value function.
    """

    new_policy = np.zeros(nS, dtype='int')

    ############################
    # YOUR IMPLEMENTATION HERE #
    for s in range(nS):
        action = 0
        q_max = 0
        for a in range(nA):
            probability, nextstate, reward, terminal = P[s][a][0]
            q = reward + gamma * probability * value_from_policy[nextstate]
            if q > q_max:
                action = a
                q_max = q
        new_policy[s] = action
    ############################
    return new_policy
Given the value function from policy improve the policy.

    Parameters
    ----------
    P, nS, nA, gamma:
        defined at beginning of file
    value_from_policy: np.ndarray
        The value calculated from the policy
    policy: np.array
        The previous policy.

    Returns
    -------
    new_policy: np.ndarray[nS]
        An array of integers. Each integer is the optimal action to take
        in that state according to the environment dynamics and the
        given value function.
    
Given the value function from policy improve the policy.
Parameters

P, nS, nA, gamma:
defined at beginning of file
value_from_policy: np.ndarray
The value calculated from the policy
policy: np.array
The previous policy.

Returns

np.ndarray[nS]
An array of integers. Each integer is the optimal action to take
in that state according to the environment dynamics and the
given value function.
Given the value function from policy improve the policy . Parameters P nS nA gamma : defined at beginning of file value_from_policy : np . ndarray The value calculated from the policy policy : np . array The previous policy . Returns np . ndarray [ nS ] An array of integers . Each integer is the optimal action to take in that state according to the environment dynamics and the given value function .
def policy_improvement(P, nS, nA, value_from_policy, policy, gamma=0.9):
    new_policy = np.zeros(nS, dtype='int')
    for s in range(nS):
        action = 0
        q_max = 0
        for a in range(nA):
            probability, nextstate, reward, terminal = P[s][a][0]
            q = reward + gamma * probability * value_from_policy[nextstate]
            if q > q_max:
                action = a
                q_max = q
        new_policy[s] = action
    return new_policy
def policy_improvement ( P , nS , nA , value_from_policy , policy , gamma = 0.9 ) : new_policy = np . zeros ( nS , dtype = 'int' ) for s in range ( nS ) : action = 0 q_max = 0 for a in range ( nA ) : probability , nextstate , reward , terminal = P [ s ] [ a ] [ 0 ] q = reward + gamma * probability * value_from_policy [ nextstate ] if q > q_max : action = a q_max = q new_policy [ s ] = action return new_policy
Given the value function from policy improve the policy.
Given the value function from policy improve the policy .
"""Given the value function from policy improve the policy.

    Parameters
    ----------
    P, nS, nA, gamma:
        defined at beginning of file
    value_from_policy: np.ndarray
        The value calculated from the policy
    policy: np.array
        The previous policy.

    Returns
    -------
    new_policy: np.ndarray[nS]
        An array of integers. Each integer is the optimal action to take
        in that state according to the environment dynamics and the
        given value function.
    """ ############################ # YOUR IMPLEMENTATION HERE # ############################
P
nS
nA
value_from_policy
policy
gamma