class_docstr: A simple out-of-graph Replay Buffer.
  
  Stores transitions, state, action, reward, next_state, terminal (and any
  extra contents specified) in a circular buffer and provides a uniform
  transition sampling function.
  
  When the states consist of stacks of observations storing the states is
  inefficient. This class writes observations and constructs the stacked states
  at sample time.
  
  Attributes:
    add_count: int, counter of how many transitions have been added (including
      the blank ones at the beginning of an episode).
    invalid_range: np.array, an array with the indices of cursor-related invalid
      transitions
    episode_end_indices: set[int], a set of indices corresponding to the
      end of an episode.
docstr: Adds a transition to the replay memory.
  
  This function checks the types and handles the padding at the beginning of
  an episode. Then it calls the _add function.
  
  Since the next_observation in the transition will be the observation added
  next there is no need to pass it.
  
  If the replay memory is at capacity the oldest transition will be discarded.
  
  Args:
    observation: np.array with shape observation_shape.
    action: int, the action in the transition.
    reward: float, the reward received in the transition.
    terminal: np.dtype, acts as a boolean indicating whether the transition
              was terminal (1) or not (0).
    *args: extra contents with shapes and dtypes according to
      extra_storage_types.
    priority: float, unused in the circular replay buffer, but may be used
      in child classes like PrioritizedReplayBuffer.
    episode_end: bool, whether this experience is the last experience in
      the episode. This is useful for tasks that terminate due to time-out,
      but do not end on a terminal state. Overloading 'terminal' may not
      be sufficient in this case, since 'terminal' is passed to the agent
      for training. 'episode_end' allows the replay buffer to determine
      episode boundaries without passing that information to the agent.
docstr_tok: ["Adds","a","transition","to","the","replay","memory",".","This","function","checks","the","types","and","handles","the","padding","at","the","beginning","of","an","episode",".","Then","it","calls","the","_add","function",".","Since","the","next_observation","in","the","transition","will","be","the","observation","added","next","there","is","no","need","to","pass","it",".","If","the","replay","memory","is","at","capacity","the","oldest","transition","will","be","discarded",".","Args",":","observation",":","np.array","with","shape","observation_shape",".","action",":","int",",","the","action","in","the","transition",".","reward",":","float",",","the","reward","received","in","the","transition",".","terminal",":","np.dtype",",","acts","as","a","boolean","indicating","whether","the","transition","was","terminal","(","1",")","or","not","(","0",")",".","*","args",":","extra","contents","with","shapes","and","dtypes","according","to","extra_storage_types",".","priority",":","float",",","unused","in","the","circular","replay","buffer",",","but","may","be","used","in","child","classes","like","PrioritizedReplayBuffer",".","episode_end",":","bool",",","whether","this","experience","is","the","last","experience","in","the","episode",".","This","is","useful","for","tasks","that","terminate","due","to","time-out",",","but","do","not","end","on","a","terminal","state",".","Overloading","'terminal","'","may","not","be","sufficient","in","this","case",",","since","'terminal","'","is","passed","to","the","agent","for","training",".","'episode_end","'","allows","the","replay","buffer","to","determine","episode","boundaries","without","passing","that","information","to","the","agent","."]
code: def add(
      self,
      observation,
      action,
      reward,
      terminal,
      *args,
      priority=None,
      episode_end=False,
  ):
      """Adds a transition to the replay memory.
  
      This function checks the types and handles the padding at the beginning of
      an episode. Then it calls the _add function.
  
      Since the next_observation in the transition will be the observation added
      next there is no need to pass it.
  
      If the replay memory is at capacity the oldest transition will be discarded.
  
      Args:
        observation: np.array with shape observation_shape.
        action: int, the action in the transition.
        reward: float, the reward received in the transition.
        terminal: np.dtype, acts as a boolean indicating whether the transition
                  was terminal (1) or not (0).
        *args: extra contents with shapes and dtypes according to
          extra_storage_types.
        priority: float, unused in the circular replay buffer, but may be used
          in child classes like PrioritizedReplayBuffer.
        episode_end: bool, whether this experience is the last experience in
          the episode. This is useful for tasks that terminate due to time-out,
          but do not end on a terminal state. Overloading 'terminal' may not
          be sufficient in this case, since 'terminal' is passed to the agent
          for training. 'episode_end' allows the replay buffer to determine
          episode boundaries without passing that information to the agent.
      """
      if priority is not None:
          args = args + (priority,)
  
      self._check_add_types(
          observation, action, reward, terminal, *args
      )
      if self._next_experience_is_episode_start:
          for _ in range(self._stack_size - 1):
              # Child classes can rely on the padding transitions being filled with
              # zeros. This is useful when there is a priority argument.
              self._add_zero_transition()
          self._next_experience_is_episode_start = False
  
      if episode_end or terminal:
          self.episode_end_indices.add(self.cursor())
          self._next_experience_is_episode_start = True
      else:
          self.episode_end_indices.discard(self.cursor())  # If present
  
      self._add(observation, action, reward, terminal, *args)
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import collections
  import gzip
  import math
  import os
  import pickle
  from absl import logging
  import gin.tf
  import numpy
  import tensorflow
label_desc: Use image node_id 7 for calling the OutOfGraphReplayBuffer obj's underlying member method code with example usage: obj.add(observation, action, reward, terminal) without return types