class_docstr: Buffer that stores (completed or truncated) episodes by their ID.
  
  Each "row" (a slot in a deque) in the buffer is occupied by one episode. If an
  incomplete episode is added to the buffer and then another chunk of that episode is
  added at a later time, the buffer will automatically concatenate the new fragment to
  the original episode. This way, episodes can be completed via subsequent `add`
  calls.
  
  Sampling returns batches of size B (number of "rows"), where each row is a
  trajectory of length T. Each trajectory contains consecutive timesteps from an
  episode, but might not start at the beginning of that episode. Should an episode end
  within such a trajectory, a random next episode (starting from its t0) will be
  concatenated to that "row". Example: `sample(B=2, T=4)` ->
  
     0 .. 1 .. 2 .. 3  <- T-axis
  0 e5   e6   e7   e8
  1 f2   f3   h0   h2
  ^ B-axis
  
  .. where e, f, and h are different (randomly picked) episodes, the 0-index (e.g. h0)
  indicates the start of an episode, and `f3` is an episode end (gym environment
  returned terminated=True or truncated=True).
  
  0-indexed returned timesteps contain the reset observation, a dummy 0.0 reward, as
  well as the first action taken in the episode (action picked after observing
  obs(0)).
  The last index in an episode (e.g. f3 in the example above) contains the final
  observation of the episode, the final reward received, a dummy action
  (repeat the previous action), as well as either terminated=True or truncated=True.
class_docstr_tok: ["Buffer","that","stores","(","completed","or","truncated",")","episodes","by","their","ID",".","Each","``","row","''","(","a","slot","in","a","deque",")","in","the","buffer","is","occupied","by","one","episode",".","If","an","incomplete","episode","is","added","to","the","buffer","and","then","another","chunk","of","that","episode","is","added","at","a","later","time",",","the","buffer","will","automatically","concatenate","the","new","fragment","to","the","original","episode",".","This","way",",","episodes","can","be","completed","via","subsequent","`","add","`","calls",".","Sampling","returns","batches","of","size","B","(","number","of","``","rows","''",")",",","where","each","row","is","a","trajectory","of","length","T.","Each","trajectory","contains","consecutive","timesteps","from","an","episode",",","but","might","not","start","at","the","beginning","of","that","episode",".","Should","an","episode","end","within","such","a","trajectory",",","a","random","next","episode","(","starting","from","its","t0",")","will","be","concatenated","to","that","``","row","''",".","Example",":","`","sample","(","B=2",",","T=4",")","`","-",">","0","..","1","..","2","..","3","<","-","T-axis","0","e5","e6","e7","e8","1","f2","f3","h0","h2","^","B-axis","..","where","e",",","f",",","and","h","are","different","(","randomly","picked",")","episodes",",","the","0-index","(","e.g",".","h0",")","indicates","the","start","of","an","episode",",","and","`","f3","`","is","an","episode","end","(","gym","environment","returned","terminated=True","or","truncated=True",")",".","0-indexed","returned","timesteps","contain","the","reset","observation",",","a","dummy","0.0","reward",",","as","well","as","the","first","action","taken","in","the","episode","(","action","picked","after","observing","obs","(","0",")",")",".","The","last","index","in","an","episode","(","e.g",".","f3","in","the","example","above",")","contains","the","final","observation","of","the","episode",",","the","final","reward","received",",","a","dummy","action","(","repeat","the","previous","action",")",",","as","well","as","either","terminated=True","or","truncated=True","."]
docstr: Returns a batch of size B (number of "rows"), where each row has length T.
  
  Each row contains consecutive timesteps from an episode, but might not start
  at the beginning of that episode. Should an episode end within such a
  row (trajectory), a random next episode (starting from its t0) will be
  concatenated to that row. For more details, see the docstring of the
  EpisodeReplayBuffer class.
  
  Args:
      num_items: See `batch_size_B`. For compatibility with the
          `ReplayBufferInterface` abstract base class.
      batch_size_B: The number of rows (trajectories) to return in the batch.
      batch_length_T: The length of each row (in timesteps) to return in the
          batch.
  
  Returns:
      The sampled batch (observations, actions, rewards, terminateds, truncateds)
          of dimensions [B, T, ...].
docstr_tok: ["Returns","a","batch","of","size","B","(","number","of","``","rows","''",")",",","where","each","row","has","length","T.","Each","row","contains","consecutive","timesteps","from","an","episode",",","but","might","not","start","at","the","beginning","of","that","episode",".","Should","an","episode","end","within","such","a","row","(","trajectory",")",",","a","random","next","episode","(","starting","from","its","t0",")","will","be","concatenated","to","that","row",".","For","more","details",",","see","the","docstring","of","the","EpisodeReplayBuffer","class",".","Args",":","num_items",":","See","`","batch_size_B","`",".","For","compatibility","with","the","`","ReplayBufferInterface","`","abstract","base","class",".","batch_size_B",":","The","number","of","rows","(","trajectories",")","to","return","in","the","batch",".","batch_length_T",":","The","length","of","each","row","(","in","timesteps",")","to","return","in","the","batch",".","Returns",":","The","sampled","batch","(","observations",",","actions",",","rewards",",","terminateds",",","truncateds",")","of","dimensions","[","B",",","T",",","...","]","."]
code: def sample(
      self,
      num_items: Optional[int] = None,
      *,
      batch_size_B: Optional[int] = None,
      batch_length_T: Optional[int] = None,
  ) -> SampleBatchType:
      """Returns a batch of size B (number of "rows"), where each row has length T.
  
      Each row contains consecutive timesteps from an episode, but might not start
      at the beginning of that episode. Should an episode end within such a
      row (trajectory), a random next episode (starting from its t0) will be
      concatenated to that row. For more details, see the docstring of the
      EpisodeReplayBuffer class.
  
      Args:
          num_items: See `batch_size_B`. For compatibility with the
              `ReplayBufferInterface` abstract base class.
          batch_size_B: The number of rows (trajectories) to return in the batch.
          batch_length_T: The length of each row (in timesteps) to return in the
              batch.
  
      Returns:
          The sampled batch (observations, actions, rewards, terminateds, truncateds)
              of dimensions [B, T, ...].
      """
      if num_items is not None:
          assert batch_size_B is None, (
              "Cannot call `sample()` with both `num_items` and `batch_size_B` "
              "provided! Use either one."
          )
          batch_size_B = num_items
  
      # Use our default values if no sizes/lengths provided.
      batch_size_B = batch_size_B or self.batch_size_B
      batch_length_T = batch_length_T or self.batch_length_T
  
      # Rows to return.
      observations = [[] for _ in range(batch_size_B)]
      actions = [[] for _ in range(batch_size_B)]
      rewards = [[] for _ in range(batch_size_B)]
      is_first = [[False] * batch_length_T for _ in range(batch_size_B)]
      is_last = [[False] * batch_length_T for _ in range(batch_size_B)]
      is_terminated = [
          [False] * batch_length_T for _ in range(batch_size_B)
      ]
      is_truncated = [
          [False] * batch_length_T for _ in range(batch_size_B)
      ]
  
      B = 0
      T = 0
      while B < batch_size_B:
          # Pull a new uniform random index tuple: (eps_idx, ts_in_eps_idx).
          index_tuple = self._indices[
              self.rng.integers(len(self._indices))
          ]
  
          # Compute the actual episode index (offset by the number of
          # already evicted episodes).
          episode_idx, episode_ts = (
              index_tuple[0] - self._num_episodes_evicted,
              index_tuple[1],
          )
          episode = self.episodes[episode_idx]
  
          # Starting a new chunk, set is_first to True.
          is_first[B][T] = True
  
          # Begin of new batch item (row).
          if len(rewards[B]) == 0:
              # And we are at the start of an episode: Set reward to 0.0.
              if episode_ts == 0:
                  rewards[B].append(0.0)
              # We are in the middle of an episode: Set reward to the previous
              # timestep's values.
              else:
                  rewards[B].append(episode.rewards[episode_ts - 1])
          # We are in the middle of a batch item (row). Concat next episode to this
          # row from the next episode's beginning. In other words, we never concat
          # a middle of an episode to another truncated one.
          else:
              episode_ts = 0
              rewards[B].append(0.0)
  
          observations[B].extend(episode.observations[episode_ts:])
          # Repeat last action to have the same number of actions than observations.
          actions[B].extend(episode.actions[episode_ts:])
          actions[B].append(episode.actions[-1])
          # Number of rewards are also the same as observations b/c we have the
          # initial 0.0 one.
          rewards[B].extend(episode.rewards[episode_ts:])
          assert (
              len(observations[B]) == len(actions[B]) == len(rewards[B])
          )
  
          T = min(len(observations[B]), batch_length_T)
  
          # Set is_last=True.
          is_last[B][T - 1] = True
          # If episode is terminated and we have reached the end of it, set
          # is_terminated=True.
          if episode.is_terminated and T == len(observations[B]):
              is_terminated[B][T - 1] = True
          # If episode is truncated and we have reached the end of it, set
          # is_truncated=True.
          elif episode.is_truncated and T == len(observations[B]):
              is_truncated[B][T - 1] = True
  
          # We are done with this batch row.
          if T == batch_length_T:
              # We may have overfilled this row: Clip trajectory at the end.
              observations[B] = observations[B][:batch_length_T]
              actions[B] = actions[B][:batch_length_T]
              rewards[B] = rewards[B][:batch_length_T]
              # Start filling the next row.
              B += 1
              T = 0
  
      # Update our sampled counter.
      self.sampled_timesteps += batch_size_B * batch_length_T
  
      # TODO: Return SampleBatch instead of this simpler dict.
      ret = {
          "obs": np.array(observations),
          "actions": np.array(actions),
          "rewards": np.array(rewards),
          "is_first": np.array(is_first),
          "is_last": np.array(is_last),
          "is_terminated": np.array(is_terminated),
          "is_truncated": np.array(is_truncated),
      }
  
      return ret
  
imports: from collections import deque
  import copy
  from typing import Any, Dict, List, Optional, Union
  import numpy
  from ray.rllib.env.single_agent_episode import SingleAgentEpisode
  from ray.rllib.utils.annotations import override
  from ray.rllib.utils.replay_buffers.base import ReplayBufferInterface
  from ray.rllib.utils.typing import SampleBatchType
label_desc: Use image node_id 4 for calling the EpisodeReplayBuffer obj's underlying member method code with example usage: obj.sample(num_items) and returns: ret