load.py
def rollout(agent, env, task, goal_conditioned, args, num_finished, debug=False):
    """Standard gym environment rollout.

    Adding more debugging options (enable with debug=True), such as printing
    the pose and saving the images and heatmaps. We can also run `dataset.py`
    and see goal images in the `goals_out` directory.

    :goal_conditioned: a boolean to check if we have goal-conditioning.
    :num_finished: to track how many episodes we have finished. Ignores any
        episodes drawn and then discarded due to initial states that were
        already done. Also used to sample the goal states for
        goal-conditioned policies. We have a fixed number of testing episodes
        (characterized by goal images), so `num_finished` is the identifier.

    Returns `t` to track episode length. Update (21 Aug 2020): also returns
    last_stuff=(obs,info), consistent with main.py and generate_goals.py.

    (13 Oct 2020): fixing so that we will always append stuff in the episode
    list for gt_state agents. The problem is that the first time step (start_t=1)
    wasn't saving because len(obs) = 0, but in gt_state we actually want to save.
    Otherwise, a length 1 episode will have len(episode)==0 later. It's not a huge
    deal because we still save the final info correctly, so that we can report
    correct stats, but it helps to have the initial info because that gives us the
    deltas over the starting state.
    """
    if debug:
        if not os.path.exists('tmp/'):
            os.makedirs('tmp/')
        print('')
    start_t = 0
    if args.agent in ['gt_state', 'gt_state_2_step']:
        start_t = 1
    episode = []
    total_reward = 0

    # Before task.reset(), need goal info for goal episode at idx `num_finished`.
    if goal_conditioned:
        task.goal_cond_testing = True
        path = os.path.join('goals', args.task)
        goal = {}
        goal['color'] = load(path, num_finished, 'last_color')
        goal['depth'] = load(path, num_finished, 'last_depth')
        goal['info'] = load(path, num_finished, 'last_info')

    goal_imgs = goal if goal_conditioned else None

    # Reset env and call task.reset(), len(obs)=0 but info will have stuff for gt_state.
    if goal_conditioned:
        obs = env.reset(task, last_info=goal['info'])
    else:
        obs = env.reset(task)
    info = env.info

    for t in range(start_t, task.max_steps):
        if debug and t > 0:
            act, extras = agent.act(obs, info, goal=goal_imgs, debug_imgs=True)
        else:
            act = agent.act(obs, info, goal=goal_imgs)

        # Optional debugging to save images, etc. Do before we get new obs.
        if debug and 'params' in act:
            debug_time_step(t, num_finished, obs, act, extras, goal=goal_imgs)

        # (13 Oct 2020) Ah, if gt_state, we won't save at start_t=1, so let's fix that!
        if (len(obs) > 0 and act['primitive']) or (args.agent in ['gt_state', 'gt_state_2_step']):
            episode.append((act, info)) # don't save obs
        (obs, reward, done, info) = env.step(act)

        # If goal-conditioning, additionally compute image-based metrics.
        if goal_conditioned and ('color' in obs and 'depth' in obs):
            info['image_metrics'] = goal_similarity(obs, goal_imgs)
        else:
            info['image_metrics'] = None

        if debug:
            print('  {}/{}, rew: {:0.3f}, len(epis): {}, act: {}, info: {}'.format(t,
                    task.max_steps, reward, len(episode), act['primitive'], info['extras']))
            if goal_conditioned:
                print('  goal-conditioning image metrics: {}'.format(info['image_metrics']))

        total_reward += reward
        last_obs_info = (obs, info)
        if done:
            break
    return total_reward, episode, t, last_obs_info
Standard gym environment rollout.

    Adding more debugging options (enable with debug=True), such as printing
    the pose and saving the images and heatmaps. We can also run `dataset.py`
    and see goal images in the `goals_out` directory.

    :goal_conditioned: a boolean to check if we have goal-conditioning.
    :num_finished: to track how many episodes we have finished. Ignores any
        episodes drawn and then discarded due to initial states that were
        already done. Also used to sample the goal states for
        goal-conditioned policies. We have a fixed number of testing episodes
        (characterized by goal images), so `num_finished` is the identifier.

    Returns `t` to track episode length. Update (21 Aug 2020): also returns
    last_stuff=(obs,info), consistent with main.py and generate_goals.py.

    (13 Oct 2020): fixing so that we will always append stuff in the episode
    list for gt_state agents. The problem is that the first time step (start_t=1)
    wasn't saving because len(obs) = 0, but in gt_state we actually want to save.
    Otherwise, a length 1 episode will have len(episode)==0 later. It's not a huge
    deal because we still save the final info correctly, so that we can report
    correct stats, but it helps to have the initial info because that gives us the
    deltas over the starting state.
    
Standard gym environment rollout.
Adding more debugging options (enable with debug=True), such as printing
the pose and saving the images and heatmaps. We can also run `dataset.py`
and see goal images in the `goals_out` directory.
Standard gym environment rollout . Adding more debugging options ( enable with debug = True ) such as printing the pose and saving the images and heatmaps . We can also run ` dataset . py ` and see goal images in the ` goals_out ` directory .
def rollout(agent, env, task, goal_conditioned, args, num_finished, debug=False):
    if debug:
        if not os.path.exists('tmp/'):
            os.makedirs('tmp/')
        print('')
    start_t = 0
    if args.agent in ['gt_state', 'gt_state_2_step']:
        start_t = 1
    episode = []
    total_reward = 0
    if goal_conditioned:
        task.goal_cond_testing = True
        path = os.path.join('goals', args.task)
        goal = {}
        goal['color'] = load(path, num_finished, 'last_color')
        goal['depth'] = load(path, num_finished, 'last_depth')
        goal['info'] = load(path, num_finished, 'last_info')
    goal_imgs = goal if goal_conditioned else None
    if goal_conditioned:
        obs = env.reset(task, last_info=goal['info'])
    else:
        obs = env.reset(task)
    info = env.info
    for t in range(start_t, task.max_steps):
        if debug and t > 0:
            act, extras = agent.act(obs, info, goal=goal_imgs, debug_imgs=True)
        else:
            act = agent.act(obs, info, goal=goal_imgs)
        if debug and 'params' in act:
            debug_time_step(t, num_finished, obs, act, extras, goal=goal_imgs)
        if (len(obs) > 0 and act['primitive']) or (args.agent in ['gt_state', 'gt_state_2_step']):
            episode.append((act, info)) 
        (obs, reward, done, info) = env.step(act)
        if goal_conditioned and ('color' in obs and 'depth' in obs):
            info['image_metrics'] = goal_similarity(obs, goal_imgs)
        else:
            info['image_metrics'] = None
        if debug:
            print('  {}/{}, rew: {:0.3f}, len(epis): {}, act: {}, info: {}'.format(t,
                    task.max_steps, reward, len(episode), act['primitive'], info['extras']))
            if goal_conditioned:
                print('  goal-conditioning image metrics: {}'.format(info['image_metrics']))
        total_reward += reward
        last_obs_info = (obs, info)
        if done:
            break
    return total_reward, episode, t, last_obs_info
def rollout ( agent , env , task , goal_conditioned , args , num_finished , debug = False ) : if debug : if not os . path . exists ( 'tmp/' ) : os . makedirs ( 'tmp/' ) print ( '' ) start_t = 0 if args . agent in [ 'gt_state' , 'gt_state_2_step' ] : start_t = 1 episode = [ ] total_reward = 0 if goal_conditioned : task . goal_cond_testing = True path = os . path . join ( 'goals' , args . task ) goal = { } goal [ 'color' ] = load ( path , num_finished , 'last_color' ) goal [ 'depth' ] = load ( path , num_finished , 'last_depth' ) goal [ 'info' ] = load ( path , num_finished , 'last_info' ) goal_imgs = goal if goal_conditioned else None if goal_conditioned : obs = env . reset ( task , last_info = goal [ 'info' ] ) else : obs = env . reset ( task ) info = env . info for t in range ( start_t , task . max_steps ) : if debug and t > 0 : act , extras = agent . act ( obs , info , goal = goal_imgs , debug_imgs = True ) else : act = agent . act ( obs , info , goal = goal_imgs ) if debug and 'params' in act : debug_time_step ( t , num_finished , obs , act , extras , goal = goal_imgs ) if ( len ( obs ) > 0 and act [ 'primitive' ] ) or ( args . agent in [ 'gt_state' , 'gt_state_2_step' ] ) : episode . append ( ( act , info ) ) ( obs , reward , done , info ) = env . step ( act ) if goal_conditioned and ( 'color' in obs and 'depth' in obs ) : info [ 'image_metrics' ] = goal_similarity ( obs , goal_imgs ) else : info [ 'image_metrics' ] = None if debug : print ( '  {}/{}, rew: {:0.3f}, len(epis): {}, act: {}, info: {}' . format ( t , task . max_steps , reward , len ( episode ) , act [ 'primitive' ] , info [ 'extras' ] ) ) if goal_conditioned : print ( '  goal-conditioning image metrics: {}' . format ( info [ 'image_metrics' ] ) ) total_reward += reward last_obs_info = ( obs , info ) if done : break return total_reward , episode , t , last_obs_info
Standard gym environment rollout.
Standard gym environment rollout .
"""Standard gym environment rollout.

    Adding more debugging options (enable with debug=True), such as printing
    the pose and saving the images and heatmaps. We can also run `dataset.py`
    and see goal images in the `goals_out` directory.

    :goal_conditioned: a boolean to check if we have goal-conditioning.
    :num_finished: to track how many episodes we have finished. Ignores any
        episodes drawn and then discarded due to initial states that were
        already done. Also used to sample the goal states for
        goal-conditioned policies. We have a fixed number of testing episodes
        (characterized by goal images), so `num_finished` is the identifier.

    Returns `t` to track episode length. Update (21 Aug 2020): also returns
    last_stuff=(obs,info), consistent with main.py and generate_goals.py.

    (13 Oct 2020): fixing so that we will always append stuff in the episode
    list for gt_state agents. The problem is that the first time step (start_t=1)
    wasn't saving because len(obs) = 0, but in gt_state we actually want to save.
    Otherwise, a length 1 episode will have len(episode)==0 later. It's not a huge
    deal because we still save the final info correctly, so that we can report
    correct stats, but it helps to have the initial info because that gives us the
    deltas over the starting state.
    """ # Before task.reset(), need goal info for goal episode at idx `num_finished`. # Reset env and call task.reset(), len(obs)=0 but info will have stuff for gt_state. # Optional debugging to save images, etc. Do before we get new obs. # (13 Oct 2020) Ah, if gt_state, we won't save at start_t=1, so let's fix that! # don't save obs # If goal-conditioning, additionally compute image-based metrics.
agent
env
task
goal_conditioned
args
num_finished
debug
goal_conditioned
a boolean to check if we have goal-conditioning.
a boolean to check if we have goal - conditioning .
num_finished
to track how many episodes we have finished. Ignores any
episodes drawn and then discarded due to initial states that were
already done. Also used to sample the goal states for
goal-conditioned policies. We have a fixed number of testing episodes
(characterized by goal images), so `num_finished` is the identifier.

Returns `t` to track episode length. Update (21 Aug 2020): also returns
last_stuff=(obs,info), consistent with main.py and generate_goals.py.

(13 Oct 2020): fixing so that we will always append stuff in the episode
list for gt_state agents. The problem is that the first time step (start_t=1)
wasn't saving because len(obs) = 0, but in gt_state we actually want to save.
Otherwise, a length 1 episode will have len(episode)==0 later. It's not a huge
deal because we still save the final info correctly, so that we can report
correct stats, but it helps to have the initial info because that gives us the
deltas over the starting state.
to track how many episodes we have finished . Ignores any episodes drawn and then discarded due to initial states that were already done . Also used to sample the goal states for goal - conditioned policies . We have a fixed number of testing episodes ( characterized by goal images ) so ` num_finished ` is the identifier . Returns ` t ` to track episode length . Update ( 21 Aug 2020 ) : also returns last_stuff = ( obs info ) consistent with main . py and generate_goals . py . ( 13 Oct 2020 ) : fixing so that we will always append stuff in the episode list for gt_state agents . The problem is that the first time step ( start_t = 1 ) wasn ' t saving because len ( obs ) = 0 but in gt_state we actually want to save . Otherwise a length 1 episode will have len ( episode ) == 0 later . It ' s not a huge deal because we still save the final info correctly so that we can report correct stats but it helps to have the initial info because that gives us the deltas over the starting state .