class_docstr: An episodic ReplayBuffer with Uniform sampling.
class_docstr_tok: ["An","episodic","ReplayBuffer","with","Uniform","sampling","."]
docstr: Creates an EpisodicReplayBuffer.
  
  This class receives a dataspec and capacity and creates a replay buffer
  supporting read/write operations, organized into episodes.
  This uses an underlying EpisodicTable with capacity equal to
  capacity.  Each row in the table can have an episode of unbounded
  length.
  
  Args:
    data_spec: A TensorSpec or a list/tuple/nest of TensorSpecs describing a
      single item that can be stored in this buffer.
    capacity: An integer, the maximum number of episodes.
    completed_only: Scalar bool.  Whether to sample full episodes (in
      as_dataset if `num_steps = None`), and whether to sample subsets from
      full episodes (in as_dataset `num_steps != None`).
    buffer_size: How many episode IDs to precalculate in a buffer when using
      `as_dataset(..., single_deterministic_pass=False)`.  This parameter
      controls how often episode IDs are sampled according to their lengths in
      the `tf.data.Dataset` returned by `as_dataset`.  Choosing a small number
      means episodes are sampled more frequently, which is expensive, but new
      data in the replay buffer is seen more quickly by the resulting dataset.
      Choosing a larger number means the sampling is less frequent and less
      costly, but new episodes and updated episode lengths may not be
      respected for up to `buffer_size` requests to the `Dataset` iterator.
      For example, if `buffer_size > 1` then a new episode may be added to the
      replay buffer, but this episode won't be "seen" by the `Dataset` for up
      to `buffer_size - 1` more accesses.
    name_prefix: A prefix for variable and op names created by this class.
    device: A TensorFlow device to place the Variables and ops.
    seed: optional random seed for sampling operations.
    begin_episode_fn: A function that maps batched tensors respecting
      `data_spec` to a boolean scalar or vector Tensor, indicating whether the
      given entries are the start of a new episode.  Used by `add_batch`,
      `add_sequence`, and `extend_episodes` to indicate whether the
      `episode_id` should be incremented.  Default value: ```python
      begin_episode_fn = lambda traj: traj.is_first() ```
    end_episode_fn: A function that maps batched tensors respecting
      `data_spec` to a boolean scalar or vector Tensor, indicating whether the
      given entries are the end of an episode.  **NOTE** The current default
      behavior is to mark an episode as completed once it receives its final
      reward.  However, additional frames may be added to the episode after
      this. Typically, exactly one boundary frame (LAST -> FIRST), is likely
      to be added to the episode after it is marked as completed).  Used by
      `add_batch`, `add_sequence`, and `extend_episodes` to indicate whether
      to end the episode at `episode_id`.  Default value: ```python
      begin_episode_fn = lambda traj: traj.is_last() ```
    dataset_drop_remainder: If `True`, then when calling `as_dataset` with
      arguments `sample_batch_size is not None`, the final batch will be
      dropped if it does not contain exactly `sample_batch_size` items.  This
      is helpful for static shape inference as the resulting tensors will
      always have leading dimension `sample_batch_size` instead of `None`.
    dataset_window_shift: Window shift used when calling `as_dataset` with
      arguments `single_deterministic_pass=True` and `num_steps is not None`.
      This determines how the resulting frames are windowed.  If `None`, then
      there is no overlap created between frames and each frame is seen
      exactly once.  For example, if `max_length=5`, `num_steps=2`,
      `sample_batch_size=None`, and `dataset_window_shift=None`, then the
      datasets returned will have frames `{[0, 1], [2, 3], [4]}`.  If
      `num_steps is not None`, then windows are created with a window overlap
      of `dataset_window_shift` and you will see each frame up to `num_steps`
      times.  For example, if `max_length=5`, `num_steps=2`,
      `sample_batch_size=None`, and `dataset_window_shift=1`, then the
      datasets returned will have windows of shifted repeated frames: `{[0,
      1], [1, 2], [2, 3], [3, 4], [4, 5]}`.  For more details, see the
      documentation of `tf.data.Dataset.window`, specifically for the `shift`
      argument.  The default behavior is to not overlap frames
      (`dataset_window_shift=None`) but users often want to see all
      combinations of frame sequences, in which case `dataset_window_shift=1`
      is the appropriate value.
code: def __init__(
      self,
      data_spec,
      capacity=1000,
      completed_only=False,
      buffer_size=8,
      name_prefix="EpisodicReplayBuffer",
      device="cpu:*",
      seed=None,
      begin_episode_fn=None,
      end_episode_fn=None,
      dataset_drop_remainder=False,
      dataset_window_shift=None,
  ):
      """Creates an EpisodicReplayBuffer.
  
      This class receives a dataspec and capacity and creates a replay buffer
      supporting read/write operations, organized into episodes.
      This uses an underlying EpisodicTable with capacity equal to
      capacity.  Each row in the table can have an episode of unbounded
      length.
  
      Args:
        data_spec: A TensorSpec or a list/tuple/nest of TensorSpecs describing a
          single item that can be stored in this buffer.
        capacity: An integer, the maximum number of episodes.
        completed_only: Scalar bool.  Whether to sample full episodes (in
          as_dataset if `num_steps = None`), and whether to sample subsets from
          full episodes (in as_dataset `num_steps != None`).
        buffer_size: How many episode IDs to precalculate in a buffer when using
          `as_dataset(..., single_deterministic_pass=False)`.  This parameter
          controls how often episode IDs are sampled according to their lengths in
          the `tf.data.Dataset` returned by `as_dataset`.  Choosing a small number
          means episodes are sampled more frequently, which is expensive, but new
          data in the replay buffer is seen more quickly by the resulting dataset.
          Choosing a larger number means the sampling is less frequent and less
          costly, but new episodes and updated episode lengths may not be
          respected for up to `buffer_size` requests to the `Dataset` iterator.
          For example, if `buffer_size > 1` then a new episode may be added to the
          replay buffer, but this episode won't be "seen" by the `Dataset` for up
          to `buffer_size - 1` more accesses.
        name_prefix: A prefix for variable and op names created by this class.
        device: A TensorFlow device to place the Variables and ops.
        seed: optional random seed for sampling operations.
        begin_episode_fn: A function that maps batched tensors respecting
          `data_spec` to a boolean scalar or vector Tensor, indicating whether the
          given entries are the start of a new episode.  Used by `add_batch`,
          `add_sequence`, and `extend_episodes` to indicate whether the
          `episode_id` should be incremented.  Default value: ```python
          begin_episode_fn = lambda traj: traj.is_first() ```
        end_episode_fn: A function that maps batched tensors respecting
          `data_spec` to a boolean scalar or vector Tensor, indicating whether the
          given entries are the end of an episode.  **NOTE** The current default
          behavior is to mark an episode as completed once it receives its final
          reward.  However, additional frames may be added to the episode after
          this. Typically, exactly one boundary frame (LAST -> FIRST), is likely
          to be added to the episode after it is marked as completed).  Used by
          `add_batch`, `add_sequence`, and `extend_episodes` to indicate whether
          to end the episode at `episode_id`.  Default value: ```python
          begin_episode_fn = lambda traj: traj.is_last() ```
        dataset_drop_remainder: If `True`, then when calling `as_dataset` with
          arguments `sample_batch_size is not None`, the final batch will be
          dropped if it does not contain exactly `sample_batch_size` items.  This
          is helpful for static shape inference as the resulting tensors will
          always have leading dimension `sample_batch_size` instead of `None`.
        dataset_window_shift: Window shift used when calling `as_dataset` with
          arguments `single_deterministic_pass=True` and `num_steps is not None`.
          This determines how the resulting frames are windowed.  If `None`, then
          there is no overlap created between frames and each frame is seen
          exactly once.  For example, if `max_length=5`, `num_steps=2`,
          `sample_batch_size=None`, and `dataset_window_shift=None`, then the
          datasets returned will have frames `{[0, 1], [2, 3], [4]}`.  If
          `num_steps is not None`, then windows are created with a window overlap
          of `dataset_window_shift` and you will see each frame up to `num_steps`
          times.  For example, if `max_length=5`, `num_steps=2`,
          `sample_batch_size=None`, and `dataset_window_shift=1`, then the
          datasets returned will have windows of shifted repeated frames: `{[0,
          1], [1, 2], [2, 3], [3, 4], [4, 5]}`.  For more details, see the
          documentation of `tf.data.Dataset.window`, specifically for the `shift`
          argument.  The default behavior is to not overlap frames
          (`dataset_window_shift=None`) but users often want to see all
          combinations of frame sequences, in which case `dataset_window_shift=1`
          is the appropriate value.
      """
      super(EpisodicReplayBuffer, self).__init__(
          data_spec, capacity, stateful_dataset=True
      )
      # Create tables `table_fn(data_spec, capacity)` that
      # can read/write nested tensors.
      table_fn = table.Table
  
      # Create tables `episode_table_fn(data_spec, capacity, name_prefix)`
      # that can read/write nested variable-length episodes
      # (in practice this means using TensorLists).
      episode_table_fn = episodic_table.EpisodicTable
  
      if begin_episode_fn is None:
  
          def _begin_episode_fn(t):
              is_first = getattr(t, "is_first", None)
              if not callable(is_first):
                  raise TypeError(
                      "Argument t is not a Trajectory; did you forget to pass the "
                      "proper begin_episode_fn to EpisodicReplayBuffer?  Saw: '{}'".format(
                          t
                      )
                  )
              return is_first()
  
          begin_episode_fn = _begin_episode_fn
      if end_episode_fn is None:
  
          def _end_episode_fn(t):
              is_last = getattr(t, "is_last", None)
              if not callable(is_last):
                  raise TypeError(
                      "Argument t is not a Trajectory; did you forget to pass the "
                      "proper end_episode_fn to EpisodicReplayBuffer?  Saw: '{}'".format(
                          t
                      )
                  )
              return is_last()
  
          end_episode_fn = _end_episode_fn
      if not callable(begin_episode_fn):
          raise TypeError(
              "begin_episode_fn is not callable: {}".format(
                  begin_episode_fn
              )
          )
      if not callable(end_episode_fn):
          raise TypeError(
              "end_episode_fn is not callable: {}".format(
                  end_episode_fn
              )
          )
      self._begin_episode_fn = begin_episode_fn
      self._end_episode_fn = end_episode_fn
      self._id_spec = specs.TensorSpec([], dtype=tf.int64, name="id")
      self._name_prefix = name_prefix
      self._device = device
      self._seed = seed
      self._completed_only = completed_only
      self._buffer_size = buffer_size
      self._dataset_window_shift = dataset_window_shift
      self._dataset_drop_remainder = dataset_drop_remainder
      self._num_writes = common.create_variable("num_writes_counter")
  
      with tf.device(self._device):
          self._data_table = episode_table_fn(
              self._data_spec, self._capacity, self._name_prefix
          )
          # The episode ids
          self._id_table = table_fn(self._id_spec, self._capacity)
          self._episodes_loc_to_id_map = common.create_variable(
              "episodes_loc_to_id_map",
              shape=[self._capacity],
              initial_value=_INVALID_EPISODE_ID,
          )
          self._episode_lengths = common.create_variable(
              "episode_lengths", shape=[self._capacity], initial_value=0
          )
          # Marks episodes as completed or not.
          # TODO(b/80430723) Add a way for users to mark episodes completed.
          # TODO(b/76154485) Change to tf.bool.
          self._episode_completed = common.create_variable(
              "episode_completed",
              shape=[self._capacity],
              dtype=tf.uint8,
              initial_value=0,
          )
          # The last episode id so far in the table.
          self._last_episode = common.create_variable(
              "last_episode", initial_value=_INVALID_EPISODE_ID
          )
          self._add_episode_critical_section = tf.CriticalSection(
              name="add_episode"
          )
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import collections
  import gin
  import tensorflow
  from tf_agents import specs
  from tf_agents.replay_buffers import episodic_table
  from tf_agents.replay_buffers import replay_buffer
  from tf_agents.replay_buffers import table
  from tf_agents.utils import common
  from tensorflow.python.data.util import nest
  from tensorflow.python.layers import utils
  from tensorflow.python.ops import list_ops
  from tensorflow.python.ops.distributions import util
label_desc: Use image node_id 1 to create a new EpisodicReplayBuffer object from inherited base classes: replay_buffer_base with example: obj = EpisodicReplayBuffer(data_spec, capacity, completed_only, buffer_size, name_prefix, device, seed, begin_episode_fn, end_episode_fn, dataset_drop_remainder, dataset_window_shift)