class_docstr: Stores multi-agent episode data.
  
  The central attribute of the class is the timestep mapping
  `global_t_to_local_t` that maps the global (environment)
  timestep to the local (agent) timesteps.
  
  The `MultiAgentEpisode` is based on the `SingleAgentEpisode`s
  for each agent, stored in `MultiAgentEpisode.agent_episodes`.
class_docstr_tok: ["Stores","multi-agent","episode","data",".","The","central","attribute","of","the","class","is","the","timestep","mapping","`","global_t_to_local_t","`","that","maps","the","global","(","environment",")","timestep","to","the","local","(","agent",")","timesteps",".","The","`","MultiAgentEpisode","`","is","based","on","the","`","SingleAgentEpisode","`","s","for","each","agent",",","stored","in","`","MultiAgentEpisode.agent_episodes","`","."]
docstr: Initializes a `MultiAgentEpisode`.
  
  Args:
      id_: Optional. Either a string to identify an episode or None.
          If None, a hexadecimal id is created. In case of providing
          a string, make sure that it is unique, as episodes get
          concatenated via this string.
      agent_ids: A list of strings containing the agent ids.
          These have to be provided at initialization.
      agent_episode_ids: Either a dictionary mapping agent ids
          corresponding `SingleAgentEpisode` or None. If None, each
          `SingleAgentEpisode` in `MultiAgentEpisode.agent_episodes`
          will generate a hexadecimal code. If a dictionary is provided
          make sure that ids are unique as agents'  `SingleAgentEpisode`s
          get concatenated or recreated by it.
      observations: A list of dictionaries mapping agent IDs to observations.
          Can be None. If provided, should match all other episode data
          (actions, rewards, etc.) in terms of list lengths and agent IDs.
      infos: A list of dictionaries mapping agent IDs to info dicts.
          Can be None. If provided, should match all other episode data
          (observations, rewards, etc.) in terms of list lengths and agent IDs.
      actions: A list of dictionaries mapping agent IDs to actions.
          Can be None. If provided, should match all other episode data
          (observations, rewards, etc.) in terms of list lengths and agent IDs.
      rewards: A list of dictionaries mapping agent IDs to rewards.
          Can be None. If provided, should match all other episode data
          (actions, rewards, etc.) in terms of list lengths and agent IDs.
      terminateds: A boolean defining if an environment has
          terminated OR a MultiAgentDict mapping individual agent ids
          to boolean flags indicating whether individual agents have terminated.
          A special __all__ key in these dicts indicates, whether the episode
          is terminated for all agents.
          The default is `False`, i.e. the episode has not been terminated.
      truncateds: A boolean defining if the environment has been
          truncated OR a MultiAgentDict mapping individual agent ids
          to boolean flags indicating whether individual agents have been
          truncated. A special __all__ key in these dicts indicates, whether the
          episode is truncated for all agents.
          The default is `False`, i.e. the episode has not been truncated.
      render_images: A list of RGB uint8 images from rendering
          the multi-agent environment.
      extra_model_outputs: A list of dictionaries mapping agent IDs to their
          corresponding extra model outputs. Each of these "outputs" is a dict
          mapping keys (str) to model output values, for example for
          `key=STATE_OUT`, the values would be the internal state outputs for
          that agent.
      t_started: The timestep (int) that defines the starting point
          of the episode. This is only larger zero, if an ongoing episode is
          created, for example by slicing an ongoing episode or by calling
          the `cut()` method on an ongoing episode.
code: def __init__(
      self,
      id_: Optional[str] = None,
      agent_ids: List[str] = None,
      agent_episode_ids: Optional[Dict[str, str]] = None,
      *,
      observations: Optional[List[MultiAgentDict]] = None,
      infos: Optional[List[MultiAgentDict]] = None,
      actions: Optional[List[MultiAgentDict]] = None,
      rewards: Optional[List[MultiAgentDict]] = None,
      terminateds: Union[MultiAgentDict, bool] = False,
      truncateds: Union[MultiAgentDict, bool] = False,
      render_images: Optional[List[np.ndarray]] = None,
      extra_model_outputs: Optional[List[MultiAgentDict]] = None,
      t_started: Optional[int] = None,
  ) -> "MultiAgentEpisode":
      """Initializes a `MultiAgentEpisode`.
  
      Args:
          id_: Optional. Either a string to identify an episode or None.
              If None, a hexadecimal id is created. In case of providing
              a string, make sure that it is unique, as episodes get
              concatenated via this string.
          agent_ids: A list of strings containing the agent ids.
              These have to be provided at initialization.
          agent_episode_ids: Either a dictionary mapping agent ids
              corresponding `SingleAgentEpisode` or None. If None, each
              `SingleAgentEpisode` in `MultiAgentEpisode.agent_episodes`
              will generate a hexadecimal code. If a dictionary is provided
              make sure that ids are unique as agents'  `SingleAgentEpisode`s
              get concatenated or recreated by it.
          observations: A list of dictionaries mapping agent IDs to observations.
              Can be None. If provided, should match all other episode data
              (actions, rewards, etc.) in terms of list lengths and agent IDs.
          infos: A list of dictionaries mapping agent IDs to info dicts.
              Can be None. If provided, should match all other episode data
              (observations, rewards, etc.) in terms of list lengths and agent IDs.
          actions: A list of dictionaries mapping agent IDs to actions.
              Can be None. If provided, should match all other episode data
              (observations, rewards, etc.) in terms of list lengths and agent IDs.
          rewards: A list of dictionaries mapping agent IDs to rewards.
              Can be None. If provided, should match all other episode data
              (actions, rewards, etc.) in terms of list lengths and agent IDs.
          terminateds: A boolean defining if an environment has
              terminated OR a MultiAgentDict mapping individual agent ids
              to boolean flags indicating whether individual agents have terminated.
              A special __all__ key in these dicts indicates, whether the episode
              is terminated for all agents.
              The default is `False`, i.e. the episode has not been terminated.
          truncateds: A boolean defining if the environment has been
              truncated OR a MultiAgentDict mapping individual agent ids
              to boolean flags indicating whether individual agents have been
              truncated. A special __all__ key in these dicts indicates, whether the
              episode is truncated for all agents.
              The default is `False`, i.e. the episode has not been truncated.
          render_images: A list of RGB uint8 images from rendering
              the multi-agent environment.
          extra_model_outputs: A list of dictionaries mapping agent IDs to their
              corresponding extra model outputs. Each of these "outputs" is a dict
              mapping keys (str) to model output values, for example for
              `key=STATE_OUT`, the values would be the internal state outputs for
              that agent.
          t_started: The timestep (int) that defines the starting point
              of the episode. This is only larger zero, if an ongoing episode is
              created, for example by slicing an ongoing episode or by calling
              the `cut()` method on an ongoing episode.
      """
  
      self.id_: str = id_ or uuid.uuid4().hex
  
      # Agent ids must be provided if data is provided. The Episode cannot
      # know how many agents are in the environment. Also the number of agents
      # can grwo or shrink.
      self._agent_ids: Union[Set[str], Set[object]] = (
          set() if agent_ids is None else set(agent_ids)
      )
  
      # The global last timestep of the episode and the timesteps when this chunk
      # started.
      self.t = self.t_started = (
          t_started
          if t_started is not None
          else (
              len(observations) - 1 if observations is not None else 0
          )
      )
  
      self.ts_carriage_return = self.t_started - (
          len(observations) - 1 if observations else 0
      )
      # Keeps track of the correspondence between agent steps and environment steps.
      # This is a mapping from agents to `_IndexMapping`. The latter keeps
      # track of the global timesteps at which an agent stepped.
      # Note, global (env) timesteps are values, while local (agent) steps are the
      # indices at which these global steps are recorded.
      self.global_t_to_local_t: Dict[
          str, List[int]
      ] = self._generate_ts_mapping(observations)
  
      # In the `MultiAgentEpisode` we need these buffers to keep track of actions,
      # that happen when an agent got observations and acted, but did not receive
      # a next observation, yet. In this case we buffer the action, add the rewards,
      # and record `is_terminated/is_truncated` until the next observation is
      # received.
      self.agent_buffers = {
          agent_id: {
              "actions": Queue(maxsize=1),
              "rewards": Queue(maxsize=1),
              "extra_model_outputs": Queue(maxsize=1),
          }
          for agent_id in self._agent_ids
      }
      # Initialize buffers.
      for buffer in self.agent_buffers.values():
          # Default reward for accumulation is zero.
          buffer["rewards"].put_nowait(0.0)
          # Default extra_mdoel_output is None.
          buffer["extra_model_outputs"].put_nowait(None)
  
      # This is needed to reconstruct global action structures for
      # orphane actions, i.e. actions that miss a 'next observation' at
      # their occurrence.
      self.global_actions_t = self._generate_global_actions_t(actions)
      # These are needed to reconstruct (see `self.get_rewards()`) reward
      # structures if agents received rewards without observations. This
      # is specific to multi-agent environemnts.
      self.partial_rewards = {
          agent_id: [] for agent_id in self._agent_ids
      }
      self.partial_rewards_t = {
          agent_id: _IndexMapping() for agent_id in self._agent_ids
      }
  
      # If this is an ongoing episode than the last `__all__` should be `False`
      self.is_terminated: bool = (
          terminateds
          if isinstance(terminateds, bool)
          else terminateds.get("__all__", False)
      )
  
      # If this is an ongoing episode than the last `__all__` should be `False`
      self.is_truncated: bool = (
          truncateds
          if isinstance(truncateds, bool)
          else truncateds.get("__all__", False)
      )
  
      # Note that all attributes will be recorded along the global timestep
      # in an multi-agent environment. `SingleAgentEpisodes`
      self.agent_episodes: MultiAgentDict = {
          agent_id: self._generate_single_agent_episode(
              agent_id,
              agent_episode_ids,
              observations,
              actions,
              rewards,
              infos,
              terminateds,
              truncateds,
              extra_model_outputs,
          )
          for agent_id in self._agent_ids
      }
  
      # RGB uint8 images from rendering the env; the images include the corresponding
      # rewards.
      assert render_images is None or observations is not None
      self.render_images: Union[List[np.ndarray], List[object]] = (
          [] if render_images is None else render_images
      )
  
imports: from collections import defaultdict
  from queue import Queue
  from typing import Any, Dict, List, Optional, Set, Union
  import uuid
  import numpy
  from ray.rllib.env.single_agent_episode import SingleAgentEpisode
  from ray.rllib.policy.sample_batch import MultiAgentBatch
  from ray.rllib.utils.typing import MultiAgentDict
label_desc: Use image node_id 1 to create a new MultiAgentEpisode object  with example: obj = MultiAgentEpisode(id_, agent_ids, agent_episode_ids)