docstr: Args:
      state_shape (tuple[int]): shape (without history) of state
      dtype: numpy dtype for the state
code: def __init__(self, max_size, state_shape, history_len, dtype="uint8"):
      """
      Args:
          state_shape (tuple[int]): shape (without history) of state
          dtype: numpy dtype for the state
      """
      self.max_size = int(max_size)
      self.state_shape = state_shape
      assert len(state_shape) in [1, 2, 3], state_shape
      # self._output_shape = self.state_shape + (history_len + 1, )
      self.history_len = int(history_len)
      self.dtype = dtype
  
      all_state_shape = (self.max_size,) + state_shape
      logger.info(
          "Creating experience replay buffer of {:.1f} GB ... "
          "use a smaller buffer if you don't have enough CPU memory.".format(
              np.prod(all_state_shape) / 1024.0**3
          )
      )
      self.state = np.zeros(all_state_shape, dtype=self.dtype)
      self.action = np.zeros((self.max_size,), dtype="int32")
      self.reward = np.zeros((self.max_size,), dtype="float32")
      self.isOver = np.zeros((self.max_size,), dtype="bool")
  
      self._curr_size = 0
      self._curr_pos = 0
  
      self.writer_lock = threading.Lock()
  
imports: import copy
  import itertools
  import numpy
  import threading
  from collections import namedtuple
  from six.moves import queue, range
  from tensorpack.utils.concurrency import LoopThread, ShareSessionThread
  from tensorpack.callbacks.base import Callback
  from tensorpack.dataflow import DataFlow
  from tensorpack.utils import logger, get_rng, get_tqdm
  from tensorpack.utils.stats import StatCounter
label_desc: Use image node_id 1 to create a new ReplayMemory object from inherited base classes: object with example: obj = ReplayMemory(max_size, state_shape, history_len, dtype)