class_docstr: A tuple representing a batch of TimeSteps.
  
  Data type for off-policy algorithms, imitation learning and batch-RL.
  
  Attributes:
      env_spec (EnvSpec): Specification for the environment from
          which this data was sampled.
      episode_infos (dict[str, np.ndarray]): A dict of numpy arrays
          containing the episode-level information of each episode. Each
          value of this dict should be a numpy array of shape :math:`(N,
          S^*)`. For example, in goal-conditioned reinforcement learning this
          could contain the goal state for each episode.
      observations (numpy.ndarray): Non-flattened array of observations.
          Typically has shape (batch_size, S^*) (the unflattened state space
          of the current environment).
      actions (numpy.ndarray): Non-flattened array of actions. Must
          have shape (batch_size, S^*) (the unflattened action space of the
          current environment).
      rewards (numpy.ndarray): Array of rewards of shape (batch_size, 1).
      next_observation (numpy.ndarray): Non-flattened array of next
          observations. Has shape (batch_size, S^*). next_observations[i] was
          observed by the agent after taking actions[i].
      env_infos (dict): A dict arbitrary environment state
          information.
      agent_infos (dict): A dict of arbitrary agent state information. For
          example, this may contain the hidden states from an RNN policy.
      step_types (numpy.ndarray): A numpy array of `StepType with shape (
          batch_size,) containing the time step types for all transitions in
          this batch.
  
  Raises:
      ValueError: If any of the above attributes do not conform to their
          prescribed types and shapes.
class_docstr_tok: ["A","tuple","representing","a","batch","of","TimeSteps",".","Data","type","for","off-policy","algorithms",",","imitation","learning","and","batch-RL",".","Attributes",":","env_spec","(","EnvSpec",")",":","Specification","for","the","environment","from","which","this","data","was","sampled",".","episode_infos","(","dict","[","str",",","np.ndarray","]",")",":","A","dict","of","numpy","arrays","containing","the","episode-level","information","of","each","episode",".","Each","value","of","this","dict","should","be","a","numpy","array","of","shape",":","math",":","`","(","N",",","S^","*",")","`",".","For","example",",","in","goal-conditioned","reinforcement","learning","this","could","contain","the","goal","state","for","each","episode",".","observations","(","numpy.ndarray",")",":","Non-flattened","array","of","observations",".","Typically","has","shape","(","batch_size",",","S^","*",")","(","the","unflattened","state","space","of","the","current","environment",")",".","actions","(","numpy.ndarray",")",":","Non-flattened","array","of","actions",".","Must","have","shape","(","batch_size",",","S^","*",")","(","the","unflattened","action","space","of","the","current","environment",")",".","rewards","(","numpy.ndarray",")",":","Array","of","rewards","of","shape","(","batch_size",",","1",")",".","next_observation","(","numpy.ndarray",")",":","Non-flattened","array","of","next","observations",".","Has","shape","(","batch_size",",","S^","*",")",".","next_observations","[","i","]","was","observed","by","the","agent","after","taking","actions","[","i","]",".","env_infos","(","dict",")",":","A","dict","arbitrary","environment","state","information",".","agent_infos","(","dict",")",":","A","dict","of","arbitrary","agent","state","information",".","For","example",",","this","may","contain","the","hidden","states","from","an","RNN","policy",".","step_types","(","numpy.ndarray",")",":","A","numpy","array","of","`","StepType","with","shape","(","batch_size",",",")","containing","the","time","step","types","for","all","transitions","in","this","batch",".","Raises",":","ValueError",":","If","any","of","the","above","attributes","do","not","conform","to","their","prescribed","types","and","shapes","."]
docstr: Convert the batch into a list of dictionaries.
  
  Breaks the :class:`~TimeStepBatch` into a list of single time step
  sample dictionaries. len(rewards) (or the number of discrete time step)
  dictionaries are returned
  
  Returns:
      list[dict[str, np.ndarray or dict[str, np.ndarray]]]: Keys:
          episode_infos (dict[str, np.ndarray]): A dict of numpy arrays
              containing the episode-level information of each episode.
              Each value of this dict must be a numpy array of shape
              :math:`(S^*,)`. For example, in goal-conditioned
              reinforcement learning this could contain the goal state
              for each episode.
          observations (numpy.ndarray): Non-flattened array of
              observations.
              Typically has shape (batch_size, S^*) (the unflattened
              state space
              of the current environment).
          actions (numpy.ndarray): Non-flattened array of actions. Must
              have shape (batch_size, S^*) (the unflattened action
              space of the
              current environment).
          rewards (numpy.ndarray): Array of rewards of shape (
              batch_size,) (1D array of length batch_size).
          next_observation (numpy.ndarray): Non-flattened array of next
              observations. Has shape (batch_size, S^*).
              next_observations[i] was
              observed by the agent after taking actions[i].
          env_infos (dict): A dict arbitrary environment state
              information.
          agent_infos (dict): A dict of arbitrary agent state
              information. For example, this may contain the
              hidden states from an RNN policy.
          step_types (numpy.ndarray): A numpy array of `StepType with
                  shape (batch_size,) containing the time step types for
                  all transitions in this batch.
docstr_tok: ["Convert","the","batch","into","a","list","of","dictionaries",".","Breaks","the",":","class",":","`","~TimeStepBatch","`","into","a","list","of","single","time","step","sample","dictionaries",".","len","(","rewards",")","(","or","the","number","of","discrete","time","step",")","dictionaries","are","returned","Returns",":","list","[","dict","[","str",",","np.ndarray","or","dict","[","str",",","np.ndarray","]","]","]",":","Keys",":","episode_infos","(","dict","[","str",",","np.ndarray","]",")",":","A","dict","of","numpy","arrays","containing","the","episode-level","information","of","each","episode",".","Each","value","of","this","dict","must","be","a","numpy","array","of","shape",":","math",":","`","(","S^","*",",",")","`",".","For","example",",","in","goal-conditioned","reinforcement","learning","this","could","contain","the","goal","state","for","each","episode",".","observations","(","numpy.ndarray",")",":","Non-flattened","array","of","observations",".","Typically","has","shape","(","batch_size",",","S^","*",")","(","the","unflattened","state","space","of","the","current","environment",")",".","actions","(","numpy.ndarray",")",":","Non-flattened","array","of","actions",".","Must","have","shape","(","batch_size",",","S^","*",")","(","the","unflattened","action","space","of","the","current","environment",")",".","rewards","(","numpy.ndarray",")",":","Array","of","rewards","of","shape","(","batch_size",",",")","(","1D","array","of","length","batch_size",")",".","next_observation","(","numpy.ndarray",")",":","Non-flattened","array","of","next","observations",".","Has","shape","(","batch_size",",","S^","*",")",".","next_observations","[","i","]","was","observed","by","the","agent","after","taking","actions","[","i","]",".","env_infos","(","dict",")",":","A","dict","arbitrary","environment","state","information",".","agent_infos","(","dict",")",":","A","dict","of","arbitrary","agent","state","information",".","For","example",",","this","may","contain","the","hidden","states","from","an","RNN","policy",".","step_types","(","numpy.ndarray",")",":","A","numpy","array","of","`","StepType","with","shape","(","batch_size",",",")","containing","the","time","step","types","for","all","transitions","in","this","batch","."]
code: def to_time_step_list(self) -> List[Dict[str, np.ndarray]]:
      """Convert the batch into a list of dictionaries.
  
      Breaks the :class:`~TimeStepBatch` into a list of single time step
      sample dictionaries. len(rewards) (or the number of discrete time step)
      dictionaries are returned
  
      Returns:
          list[dict[str, np.ndarray or dict[str, np.ndarray]]]: Keys:
              episode_infos (dict[str, np.ndarray]): A dict of numpy arrays
                  containing the episode-level information of each episode.
                  Each value of this dict must be a numpy array of shape
                  :math:`(S^*,)`. For example, in goal-conditioned
                  reinforcement learning this could contain the goal state
                  for each episode.
              observations (numpy.ndarray): Non-flattened array of
                  observations.
                  Typically has shape (batch_size, S^*) (the unflattened
                  state space
                  of the current environment).
              actions (numpy.ndarray): Non-flattened array of actions. Must
                  have shape (batch_size, S^*) (the unflattened action
                  space of the
                  current environment).
              rewards (numpy.ndarray): Array of rewards of shape (
                  batch_size,) (1D array of length batch_size).
              next_observation (numpy.ndarray): Non-flattened array of next
                  observations. Has shape (batch_size, S^*).
                  next_observations[i] was
                  observed by the agent after taking actions[i].
              env_infos (dict): A dict arbitrary environment state
                  information.
              agent_infos (dict): A dict of arbitrary agent state
                  information. For example, this may contain the
                  hidden states from an RNN policy.
              step_types (numpy.ndarray): A numpy array of `StepType with
                      shape (batch_size,) containing the time step types for
                      all transitions in this batch.
      """
      samples = []
      for i in range(len(self.rewards)):
          samples.append(
              {
                  "episode_infos": {
                      k: np.asarray([v[i]])
                      for (k, v) in self.episode_infos.items()
                  },
                  "observations": np.asarray([self.observations[i]]),
                  "actions": np.asarray([self.actions[i]]),
                  "rewards": np.asarray([self.rewards[i]]),
                  "next_observations": np.asarray(
                      [self.next_observations[i]]
                  ),
                  "env_infos": {
                      k: np.asarray([v[i]])
                      for (k, v) in self.env_infos.items()
                  },
                  "agent_infos": {
                      k: np.asarray([v[i]])
                      for (k, v) in self.agent_infos.items()
                  },
                  "step_types": np.asarray([self.step_types[i]]),
              }
          )
      return samples
  
imports: from dataclasses import dataclass
  import enum
  from typing import Dict, List
  import warnings
  import numpy
  from garage.np import concat_tensor_dict_list, pad_batch_array, slice_nested_dict, stack_tensor_dict_list
label_desc: Use image node_id 4 for calling the TimeStepBatch obj's underlying member method code with example usage: obj.to_time_step_list() and returns: samples