class_docstr: Wrapper of OutOfGraphReplayBuffer with an in graph sampling mechanism.
  
  Usage:
    To add a transition:  call the add function.
  
    To sample a batch:    Construct operations that depend on any of the
                          tensors is the transition dictionary. Every sess.run
                          that requires any of these tensors will sample a new
                          transition.
docstr: Initializes WrappedReplayBuffer.
  
  Args:
    observation_shape: tuple of ints.
    stack_size: int, number of frames to use in state stack.
    use_staging: bool, when True it would use a staging area to prefetch
      the next sampling batch.
    replay_capacity: int, number of transitions to keep in memory.
    batch_size: int.
    update_horizon: int, length of update ('n' in n-step update).
    gamma: int, the discount factor.
    wrapped_memory: The 'inner' memory data structure. If None,
      it creates the standard DQN replay memory.
    max_sample_attempts: int, the maximum number of attempts allowed to
      get a sample.
    extra_storage_types: list of ReplayElements defining the type of the extra
      contents that will be stored and returned by sample_transition_batch.
    observation_dtype: np.dtype, type of the observations. Defaults to
      np.uint8 for Atari 2600.
    terminal_dtype: np.dtype, type of the terminals. Defaults to np.uint8 for
      Atari 2600.
    action_shape: tuple of ints, the shape for the action vector. Empty tuple
      means the action is a scalar.
    action_dtype: np.dtype, type of elements in the action.
    reward_shape: tuple of ints, the shape of the reward vector. Empty tuple
      means the reward is a scalar.
    reward_dtype: np.dtype, type of elements in the reward.
  
  Raises:
    ValueError: If update_horizon is not positive.
    ValueError: If discount factor is not in [0, 1].
docstr_tok: ["Initializes","WrappedReplayBuffer",".","Args",":","observation_shape",":","tuple","of","ints",".","stack_size",":","int",",","number","of","frames","to","use","in","state","stack",".","use_staging",":","bool",",","when","True","it","would","use","a","staging","area","to","prefetch","the","next","sampling","batch",".","replay_capacity",":","int",",","number","of","transitions","to","keep","in","memory",".","batch_size",":","int",".","update_horizon",":","int",",","length","of","update","(","'n","'","in","n-step","update",")",".","gamma",":","int",",","the","discount","factor",".","wrapped_memory",":","The","'inner","'","memory","data","structure",".","If","None",",","it","creates","the","standard","DQN","replay","memory",".","max_sample_attempts",":","int",",","the","maximum","number","of","attempts","allowed","to","get","a","sample",".","extra_storage_types",":","list","of","ReplayElements","defining","the","type","of","the","extra","contents","that","will","be","stored","and","returned","by","sample_transition_batch",".","observation_dtype",":","np.dtype",",","type","of","the","observations",".","Defaults","to","np.uint8","for","Atari","2600.","terminal_dtype",":","np.dtype",",","type","of","the","terminals",".","Defaults","to","np.uint8","for","Atari","2600.","action_shape",":","tuple","of","ints",",","the","shape","for","the","action","vector",".","Empty","tuple","means","the","action","is","a","scalar",".","action_dtype",":","np.dtype",",","type","of","elements","in","the","action",".","reward_shape",":","tuple","of","ints",",","the","shape","of","the","reward","vector",".","Empty","tuple","means","the","reward","is","a","scalar",".","reward_dtype",":","np.dtype",",","type","of","elements","in","the","reward",".","Raises",":","ValueError",":","If","update_horizon","is","not","positive",".","ValueError",":","If","discount","factor","is","not","in","[","0",",","1","]","."]
code: def __init__(
      self,
      observation_shape,
      stack_size,
      use_staging=False,
      replay_capacity=1000000,
      batch_size=32,
      update_horizon=1,
      gamma=0.99,
      wrapped_memory=None,
      max_sample_attempts=1000,
      extra_storage_types=None,
      observation_dtype=np.uint8,
      terminal_dtype=np.uint8,
      action_shape=(),
      action_dtype=np.int32,
      reward_shape=(),
      reward_dtype=np.float32,
  ):
      """Initializes WrappedReplayBuffer.
  
      Args:
        observation_shape: tuple of ints.
        stack_size: int, number of frames to use in state stack.
        use_staging: bool, when True it would use a staging area to prefetch
          the next sampling batch.
        replay_capacity: int, number of transitions to keep in memory.
        batch_size: int.
        update_horizon: int, length of update ('n' in n-step update).
        gamma: int, the discount factor.
        wrapped_memory: The 'inner' memory data structure. If None,
          it creates the standard DQN replay memory.
        max_sample_attempts: int, the maximum number of attempts allowed to
          get a sample.
        extra_storage_types: list of ReplayElements defining the type of the extra
          contents that will be stored and returned by sample_transition_batch.
        observation_dtype: np.dtype, type of the observations. Defaults to
          np.uint8 for Atari 2600.
        terminal_dtype: np.dtype, type of the terminals. Defaults to np.uint8 for
          Atari 2600.
        action_shape: tuple of ints, the shape for the action vector. Empty tuple
          means the action is a scalar.
        action_dtype: np.dtype, type of elements in the action.
        reward_shape: tuple of ints, the shape of the reward vector. Empty tuple
          means the reward is a scalar.
        reward_dtype: np.dtype, type of elements in the reward.
  
      Raises:
        ValueError: If update_horizon is not positive.
        ValueError: If discount factor is not in [0, 1].
      """
      if replay_capacity < update_horizon + 1:
          raise ValueError(
              "Update horizon ({}) should be significantly smaller "
              "than replay capacity ({}).".format(
                  update_horizon, replay_capacity
              )
          )
      if not update_horizon >= 1:
          raise ValueError("Update horizon must be positive.")
      if not 0.0 <= gamma <= 1.0:
          raise ValueError("Discount factor (gamma) must be in [0, 1].")
  
      self.batch_size = batch_size
  
      # Mainly used to allow subclasses to pass self.memory.
      if wrapped_memory is not None:
          self.memory = wrapped_memory
      else:
          self.memory = OutOfGraphReplayBuffer(
              observation_shape,
              stack_size,
              replay_capacity,
              batch_size,
              update_horizon,
              gamma,
              max_sample_attempts,
              observation_dtype=observation_dtype,
              terminal_dtype=terminal_dtype,
              extra_storage_types=extra_storage_types,
              action_shape=action_shape,
              action_dtype=action_dtype,
              reward_shape=reward_shape,
              reward_dtype=reward_dtype,
          )
  
      self.create_sampling_ops(use_staging)
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import collections
  import gzip
  import math
  import os
  import pickle
  from absl import logging
  import gin.tf
  import numpy
  import tensorflow
label_desc: Use image node_id 1 to create a new WrappedReplayBuffer object from inherited base classes: object with example: obj = WrappedReplayBuffer(observation_shape, stack_size, use_staging, replay_capacity, batch_size, update_horizon, gamma, wrapped_memory, max_sample_attempts, extra_storage_types, observation_dtype, terminal_dtype, action_shape, action_dtype, reward_shape, reward_dtype)