class_docstr: Buffer that stores (completed or truncated) episodes by their ID.
  
  Each "row" (a slot in a deque) in the buffer is occupied by one episode. If an
  incomplete episode is added to the buffer and then another chunk of that episode is
  added at a later time, the buffer will automatically concatenate the new fragment to
  the original episode. This way, episodes can be completed via subsequent `add`
  calls.
  
  Sampling returns batches of size B (number of "rows"), where each row is a
  trajectory of length T. Each trajectory contains consecutive timesteps from an
  episode, but might not start at the beginning of that episode. Should an episode end
  within such a trajectory, a random next episode (starting from its t0) will be
  concatenated to that "row". Example: `sample(B=2, T=4)` ->
  
     0 .. 1 .. 2 .. 3  <- T-axis
  0 e5   e6   e7   e8
  1 f2   f3   h0   h2
  ^ B-axis
  
  .. where e, f, and h are different (randomly picked) episodes, the 0-index (e.g. h0)
  indicates the start of an episode, and `f3` is an episode end (gym environment
  returned terminated=True or truncated=True).
  
  0-indexed returned timesteps contain the reset observation, a dummy 0.0 reward, as
  well as the first action taken in the episode (action picked after observing
  obs(0)).
  The last index in an episode (e.g. f3 in the example above) contains the final
  observation of the episode, the final reward received, a dummy action
  (repeat the previous action), as well as either terminated=True or truncated=True.
class_docstr_tok: ["Buffer","that","stores","(","completed","or","truncated",")","episodes","by","their","ID",".","Each","``","row","''","(","a","slot","in","a","deque",")","in","the","buffer","is","occupied","by","one","episode",".","If","an","incomplete","episode","is","added","to","the","buffer","and","then","another","chunk","of","that","episode","is","added","at","a","later","time",",","the","buffer","will","automatically","concatenate","the","new","fragment","to","the","original","episode",".","This","way",",","episodes","can","be","completed","via","subsequent","`","add","`","calls",".","Sampling","returns","batches","of","size","B","(","number","of","``","rows","''",")",",","where","each","row","is","a","trajectory","of","length","T.","Each","trajectory","contains","consecutive","timesteps","from","an","episode",",","but","might","not","start","at","the","beginning","of","that","episode",".","Should","an","episode","end","within","such","a","trajectory",",","a","random","next","episode","(","starting","from","its","t0",")","will","be","concatenated","to","that","``","row","''",".","Example",":","`","sample","(","B=2",",","T=4",")","`","-",">","0","..","1","..","2","..","3","<","-","T-axis","0","e5","e6","e7","e8","1","f2","f3","h0","h2","^","B-axis","..","where","e",",","f",",","and","h","are","different","(","randomly","picked",")","episodes",",","the","0-index","(","e.g",".","h0",")","indicates","the","start","of","an","episode",",","and","`","f3","`","is","an","episode","end","(","gym","environment","returned","terminated=True","or","truncated=True",")",".","0-indexed","returned","timesteps","contain","the","reset","observation",",","a","dummy","0.0","reward",",","as","well","as","the","first","action","taken","in","the","episode","(","action","picked","after","observing","obs","(","0",")",")",".","The","last","index","in","an","episode","(","e.g",".","f3","in","the","example","above",")","contains","the","final","observation","of","the","episode",",","the","final","reward","received",",","a","dummy","action","(","repeat","the","previous","action",")",",","as","well","as","either","terminated=True","or","truncated=True","."]
docstr: Initializes an EpisodeReplayBuffer instance.
  
  Args:
      capacity: The total number of timesteps to be storable in this buffer.
          Will start ejecting old episodes once this limit is reached.
      batch_size_B: The number of rows in a SampleBatch returned from `sample()`.
      batch_length_T: The length of each row in a SampleBatch returned from
          `sample()`.
docstr_tok: ["Initializes","an","EpisodeReplayBuffer","instance",".","Args",":","capacity",":","The","total","number","of","timesteps","to","be","storable","in","this","buffer",".","Will","start","ejecting","old","episodes","once","this","limit","is","reached",".","batch_size_B",":","The","number","of","rows","in","a","SampleBatch","returned","from","`","sample","(",")","`",".","batch_length_T",":","The","length","of","each","row","in","a","SampleBatch","returned","from","`","sample","(",")","`","."]
code: def __init__(
      self,
      capacity: int = 10000,
      *,
      batch_size_B: int = 16,
      batch_length_T: int = 64,
  ):
      """Initializes an EpisodeReplayBuffer instance.
  
      Args:
          capacity: The total number of timesteps to be storable in this buffer.
              Will start ejecting old episodes once this limit is reached.
          batch_size_B: The number of rows in a SampleBatch returned from `sample()`.
          batch_length_T: The length of each row in a SampleBatch returned from
              `sample()`.
      """
      self.capacity = capacity
      self.batch_size_B = batch_size_B
      self.batch_length_T = batch_length_T
  
      # The actual episode buffer. We are using a deque here for faster insertion
      # (left side) and eviction (right side) of data.
      self.episodes = deque()
      # Maps (unique) episode IDs to the index under which to find this episode
      # within our `self.episodes` deque.
      # Note that even after eviction started, the indices in here will NOT be
      # changed. We will therefore need to offset all indices in
      # `self.episode_id_to_index` by the number of episodes that have already been
      # evicted (self._num_episodes_evicted) in order to get the actual index to use
      # on `self.episodes`.
      self.episode_id_to_index = {}
      # The number of episodes that have already been evicted from the buffer
      # due to reaching capacity.
      self._num_episodes_evicted = 0
  
      # List storing all index tuples: (eps_idx, ts_in_eps_idx), where ...
      # `eps_idx - self._num_episodes_evicted' is the index into self.episodes.
      # `ts_in_eps_idx` is the timestep index within that episode
      #  (0 = 1st timestep, etc..).
      # We sample uniformly from the set of these indices in a `sample()`
      # call.
      self._indices = []
  
      # The size of the buffer in timesteps.
      self._num_timesteps = 0
      # The number of timesteps added thus far.
      self._num_timesteps_added = 0
  
      # How many timesteps have been sampled from the buffer in total?
      self.sampled_timesteps = 0
  
      self.rng = np.random.default_rng(seed=None)
  
imports: from collections import deque
  import copy
  from typing import Any, Dict, List, Optional, Union
  import numpy
  from ray.rllib.env.single_agent_episode import SingleAgentEpisode
  from ray.rllib.utils.annotations import override
  from ray.rllib.utils.replay_buffers.base import ReplayBufferInterface
  from ray.rllib.utils.typing import SampleBatchType
label_desc: Use image node_id 1 to create a new EpisodeReplayBuffer object from inherited base classes: ReplayBufferInterface with example: obj = EpisodeReplayBuffer(capacity)