class_docstr: An EnvProblem implemented as a batch of gym envs.
  
  This implementation should work well for cases where the env is not batched by
  default ex: any gym env. In this case we create `batch_size` number of envs
  and store them in a list. Any function then that interacts with the envs, like
  reset, step or close goes over the env list to do the needful, ex: when reset
  is called with specific indices we reset only those indices, etc.
  
  The usage of this class will look like the following:
  
  # 1. Creates and initializes the env_problem.
  ep = env_problem.EnvProblem(...)
  
  # 2. One needs to call reset() at the start, this resets all envs.
  ep.reset()
  
  # 3. Call step with actions for all envs, i.e. len(action) = batch_size
  obs, rewards, dones, infos = ep.step(actions)
  
  # 4. Figure out which envs got done and reset only those.
  ep.reset(indices=env_problem_utils.done_indices(dones))
  
  # 5. Go back to Step #3 to further interact with the env or just dump the
  # generated data to disk by calling:
  ep.generate_data(...)
  
  # 6. If we now need to use this object again to play a few more iterations
  # perhaps with a different batch size or maybe not recording the data, then
  # we need to re-initialize environments and do some book-keeping, call:
  ep.initialize_environments(batch_size)
  
  # 7. Go back to Step #2, i.e. reset all envs.
  
  NOTE: Look at `EnvProblemTest.test_interaction_with_env` and/or
  `EnvProblemTest.test_generate_data`
  
  NOTE: We rely heavily that the underlying environments expose a gym style
  interface, i.e. in addition to reset(), step() and close() we have access to
  the following properties: observation_space, action_space, reward_range.
class_docstr_tok: ["An","EnvProblem","implemented","as","a","batch","of","gym","envs",".","This","implementation","should","work","well","for","cases","where","the","env","is","not","batched","by","default","ex",":","any","gym","env",".","In","this","case","we","create","`","batch_size","`","number","of","envs","and","store","them","in","a","list",".","Any","function","then","that","interacts","with","the","envs",",","like","reset",",","step","or","close","goes","over","the","env","list","to","do","the","needful",",","ex",":","when","reset","is","called","with","specific","indices","we","reset","only","those","indices",",","etc",".","The","usage","of","this","class","will","look","like","the","following",":","#","1",".","Creates","and","initializes","the","env_problem",".","ep","=","env_problem.EnvProblem","(","...",")","#","2",".","One","needs","to","call","reset","(",")","at","the","start",",","this","resets","all","envs",".","ep.reset","(",")","#","3",".","Call","step","with","actions","for","all","envs",",","i.e",".","len","(","action",")","=","batch_size","obs",",","rewards",",","dones",",","infos","=","ep.step","(","actions",")","#","4",".","Figure","out","which","envs","got","done","and","reset","only","those",".","ep.reset","(","indices=env_problem_utils.done_indices","(","dones",")",")","#","5",".","Go","back","to","Step","#","3","to","further","interact","with","the","env","or","just","dump","the","#","generated","data","to","disk","by","calling",":","ep.generate_data","(","...",")","#","6",".","If","we","now","need","to","use","this","object","again","to","play","a","few","more","iterations","#","perhaps","with","a","different","batch","size","or","maybe","not","recording","the","data",",","then","#","we","need","to","re-initialize","environments","and","do","some","book-keeping",",","call",":","ep.initialize_environments","(","batch_size",")","#","7",".","Go","back","to","Step","#","2",",","i.e",".","reset","all","envs",".","NOTE",":","Look","at","`","EnvProblemTest.test_interaction_with_env","`","and\/or","`","EnvProblemTest.test_generate_data","`","NOTE",":","We","rely","heavily","that","the","underlying","environments","expose","a","gym","style","interface",",","i.e",".","in","addition","to","reset","(",")",",","step","(",")","and","close","(",")","we","have","access","to","the","following","properties",":","observation_space",",","action_space",",","reward_range","."]
docstr: Initializes this class by creating the envs and managing trajectories.
  
  Args:
    base_env_name: (string) passed to `gym.make` to make the underlying
      environment.
    env_wrapper_fn: (callable(env): env) Applies gym wrappers to the base
      environment.
    reward_range: (tuple(number, number) or None) the first element is the
      minimum reward and the second is the maximum reward, used to clip and
      process the raw reward in `process_rewards`. If None, this is inferred
      from the inner environments.
    **kwargs: (dict) Arguments passed to the base class.
docstr_tok: ["Initializes","this","class","by","creating","the","envs","and","managing","trajectories",".","Args",":","base_env_name",":","(","string",")","passed","to","`","gym.make","`","to","make","the","underlying","environment",".","env_wrapper_fn",":","(","callable","(","env",")",":","env",")","Applies","gym","wrappers","to","the","base","environment",".","reward_range",":","(","tuple","(","number",",","number",")","or","None",")","the","first","element","is","the","minimum","reward","and","the","second","is","the","maximum","reward",",","used","to","clip","and","process","the","raw","reward","in","`","process_rewards","`",".","If","None",",","this","is","inferred","from","the","inner","environments",".","*","*","kwargs",":","(","dict",")","Arguments","passed","to","the","base","class","."]
code: def __init__(
      self,
      base_env_name=None,
      env_wrapper_fn=None,
      reward_range=None,
      **kwargs,
  ):
      """Initializes this class by creating the envs and managing trajectories.
  
      Args:
        base_env_name: (string) passed to `gym.make` to make the underlying
          environment.
        env_wrapper_fn: (callable(env): env) Applies gym wrappers to the base
          environment.
        reward_range: (tuple(number, number) or None) the first element is the
          minimum reward and the second is the maximum reward, used to clip and
          process the raw reward in `process_rewards`. If None, this is inferred
          from the inner environments.
        **kwargs: (dict) Arguments passed to the base class.
      """
      # Name for the base environment, will be used in `gym.make` in
      # the default implementation of `initialize_environments`.
      self._base_env_name = base_env_name
  
      # An env generates data when it is given actions by an agent which is either
      # a policy or a human -- this is supposed to be the `id` of the agent.
      #
      # In practice, this is used only to store (and possibly retrieve) history
      # to an appropriate directory.
      self._agent_id = "default"
  
      # We clip rewards to this range before processing them further, as described
      # in `process_rewards`.
      self._reward_range = reward_range
  
      # Initialize the environment(s).
  
      # This can either be a list of environments of len `batch_size` or this can
      # be a Neural Network, in which case it will be fed input with first
      # dimension = `batch_size`.
      self._envs = None
      self._pool = None
  
      self._env_wrapper_fn = env_wrapper_fn
  
      # Call the super's ctor. It will use some of the member fields, so we call
      # it in the end.
      super(GymEnvProblem, self).__init__(**kwargs)
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import copy
  import multiprocessing.pool
  import time
  from absl import logging
  import gym
  import numpy
  from tensor2tensor.envs import env_problem
  from tensor2tensor.envs import trajectory
label_desc: Use image node_id 1 to create a new GymEnvProblem object from inherited base classes: env_problem with example: obj = GymEnvProblem(base_env_name, env_wrapper_fn, reward_range)