class_docstr: Base class of an env which generates data like a problem class.
  
  EnvProblem is both a gym Env and a Problem, since it subclasses both.
  
  Conceptually it contains `batch_size` environments on which step (and reset)
  are called. The data that is generated by the repeated application of step and
  reset is stored within this class and is persisted on disk when we call
  `generate_data` on it.
  
  Subclasses *should* override the following functions:
  - initialize_environments
  - observation_space
  - action_space
  - reward_range
  - _reset
  - _step
  - _render
  
  In addition, they should ovveride the following functions, which are used in
  the `hparams` function to return modalities and vocab_sizes.
  - input_modality
  - input_vocab_size
  - target_modality
  - target_vocab_size
  - action_modality
  - reward_modality
  
  NON NATIVELY BATCHED ENVS:
  
  The implementation for cases where the env is not batched by default is
  `gym_env_problem.GymEnvProblem`.
  
  NATIVELY BATCHED ENVS:
  
  If however, our env is a neural network, which can be batched by default, we
  should:
  
  # 1 - Give it a gym style interface, by overriding observation_space and
  action_space.
  
  # 2 - Override `_reset` and `_step` to do the reset and step in a natively
  batched manner.
  
  # 3 - More generally any function that iterates over the self._env list will
  need to be overridden, ex: `_verify_same_spaces` and `initialize_environments`
  
  KNOWN LIMITATIONS:
  
  - observation_space and action_space should be subclasses of gym.spaces
  - not all subclasses of gym.spaces are supported
class_docstr_tok: ["Base","class","of","an","env","which","generates","data","like","a","problem","class",".","EnvProblem","is","both","a","gym","Env","and","a","Problem",",","since","it","subclasses","both",".","Conceptually","it","contains","`","batch_size","`","environments","on","which","step","(","and","reset",")","are","called",".","The","data","that","is","generated","by","the","repeated","application","of","step","and","reset","is","stored","within","this","class","and","is","persisted","on","disk","when","we","call","`","generate_data","`","on","it",".","Subclasses","*","should","*","override","the","following","functions",":","-","initialize_environments","-","observation_space","-","action_space","-","reward_range","-","_reset","-","_step","-","_render","In","addition",",","they","should","ovveride","the","following","functions",",","which","are","used","in","the","`","hparams","`","function","to","return","modalities","and","vocab_sizes",".","-","input_modality","-","input_vocab_size","-","target_modality","-","target_vocab_size","-","action_modality","-","reward_modality","NON","NATIVELY","BATCHED","ENVS",":","The","implementation","for","cases","where","the","env","is","not","batched","by","default","is","`","gym_env_problem.GymEnvProblem","`",".","NATIVELY","BATCHED","ENVS",":","If","however",",","our","env","is","a","neural","network",",","which","can","be","batched","by","default",",","we","should",":","#","1","-","Give","it","a","gym","style","interface",",","by","overriding","observation_space","and","action_space",".","#","2","-","Override","`","_reset","`","and","`","_step","`","to","do","the","reset","and","step","in","a","natively","batched","manner",".","#","3","-","More","generally","any","function","that","iterates","over","the","self._env","list","will","need","to","be","overridden",",","ex",":","`","_verify_same_spaces","`","and","`","initialize_environments","`","KNOWN","LIMITATIONS",":","-","observation_space","and","action_space","should","be","subclasses","of","gym.spaces","-","not","all","subclasses","of","gym.spaces","are","supported"]
docstr: Initializes this class by creating the envs and managing trajectories.
  
  Args:
    batch_size: (int or None) How many envs to make in the non natively
      batched mode.
    discrete_rewards: (bool) whether to round the rewards to the nearest
      integer.
    parallelism: (int) If this is greater than one then we run the envs in
      parallel using multi-threading.
    **env_kwargs: (dict) Additional kwargs to pass to the environments.
code: def __init__(
      self,
      batch_size=None,
      discrete_rewards=True,
      parallelism=1,
      **env_kwargs,
  ):
      """Initializes this class by creating the envs and managing trajectories.
  
      Args:
        batch_size: (int or None) How many envs to make in the non natively
          batched mode.
        discrete_rewards: (bool) whether to round the rewards to the nearest
          integer.
        parallelism: (int) If this is greater than one then we run the envs in
          parallel using multi-threading.
        **env_kwargs: (dict) Additional kwargs to pass to the environments.
      """
  
      # Call the super's ctor.
      problem.Problem.__init__(self, was_reversed=False, was_copy=False)
  
      # An env generates data when it is given actions by an agent which is either
      # a policy or a human -- this is supposed to be the `id` of the agent.
      #
      # In practice, this is used only to store (and possibly retrieve) history
      # to an appropriate directory.
      self._agent_id = "default"
  
      # If set, we discretize the rewards and treat them as integers.
      self._discrete_rewards = discrete_rewards
  
      # A data structure to hold the `batch_size` currently active trajectories
      # and also the ones that are completed, i.e. done.
      self._trajectories = None
  
      self._batch_size = None
  
      self._parallelism = None
      # The parallelism is passes in via env_kwargs because it will be used by
      # `GymEnvProblem` to paralellize env actions across a batch.
      env_kwargs["parallelism"] = parallelism
  
      if batch_size is not None:
          self.initialize(batch_size=batch_size, **env_kwargs)
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  from absl import logging
  from gym.core import Env
  import numpy
  import six
  from tensor2tensor.data_generators import generator_utils
  from tensor2tensor.data_generators import problem
  from tensor2tensor.envs import gym_spaces_utils
  from tensor2tensor.envs import trajectory
  from tensor2tensor.layers import modalities
  from tensor2tensor.utils import contrib
  import tensorflow.compat.v1
label_desc: Use image node_id 1 to create a new EnvProblem object from inherited base classes: Env, problem with example: obj = EnvProblem(batch_size, discrete_rewards, parallelism)