class_docstr: Abstract base class for TF-Agents replay buffer.
  
  In eager mode, methods modify the buffer or return values directly. In graph
  mode, methods return ops that do so when executed.
class_docstr_tok: ["Abstract","base","class","for","TF-Agents","replay","buffer",".","In","eager","mode",",","methods","modify","the","buffer","or","return","values","directly",".","In","graph","mode",",","methods","return","ops","that","do","so","when","executed","."]
docstr: Creates and returns a dataset that returns entries from the buffer.
  
  A single entry from the dataset is the result of the following pipeline:
  
    * Sample sequences from the underlying data store
    * (optionally) Process them with `sequence_preprocess_fn`,
    * (optionally) Split them into subsequences of length `num_steps`
    * (optionally) Batch them into batches of size `sample_batch_size`.
  
  In practice, this pipeline is executed in parallel as much as possible
  if `num_parallel_calls != 1`.
  
  Some additional notes:
  
  If `num_steps is None`, different replay buffers will behave differently.
  For example, `TFUniformReplayBuffer` will return single time steps without
  a time dimension.  In contrast, e.g., `EpisodicReplayBuffer` will return
  full sequences (since each sequence may be an episode of unknown length,
  the outermost shape dimension will be `None`).
  
  If `sample_batch_size is None`, no batching is performed; and there is no
  outer batch dimension in the returned Dataset entries.  This setting
  is useful with variable episode lengths using e.g. `EpisodicReplayBuffer`,
  because it allows the user to get full episodes back, and use `tf.data`
  to build padded or truncated batches themselves.
  
  If `single_deterministic_pass == True`, the replay buffer will make
  every attempt to ensure every time step is visited once and exactly once
  in a deterministic manner (though true determinism depends on the
  underlying data store).  Additional work may be done to ensure minibatches
  do not have multiple rows from the same episode.  In some cases, this
  may mean arguments like `num_parallel_calls` are ignored.
  
  Args:
    sample_batch_size: (Optional.) An optional batch_size to specify the
      number of items to return. If None (default), a single item is returned
      which matches the data_spec of this class (without a batch dimension).
      Otherwise, a batch of sample_batch_size items is returned, where each
      tensor in items will have its first dimension equal to sample_batch_size
      and the rest of the dimensions match the corresponding data_spec.
    num_steps: (Optional.)  Optional way to specify that sub-episodes are
      desired. If None (default), a batch of single items is returned.
      Otherwise, a batch of sub-episodes is returned, where a sub-episode is a
      sequence of consecutive items in the replay_buffer. The returned tensors
      will have first dimension equal to sample_batch_size (if
      sample_batch_size is not None), subsequent dimension equal to num_steps,
      and remaining dimensions which match the data_spec of this class.
    num_parallel_calls: (Optional.) A `tf.int32` scalar `tf.Tensor`,
      representing the number elements to process in parallel. If not
      specified, elements will be processed sequentially.
    sequence_preprocess_fn: (Optional) fn for preprocessing the collected data
      before it is split into subsequences of length `num_steps`. Defined in
      `TFAgent.preprocess_sequence`.  Defaults to pass through.
    single_deterministic_pass: Python boolean.  If `True`, the dataset will
      return a single deterministic pass through its underlying data.
      **NOTE**: If the buffer is modified while a Dataset iterator is
      iterating over this data, the iterator may miss any new data or
      otherwise have subtly invalid data.
  
  Returns:
    A dataset of type tf.data.Dataset, elements of which are 2-tuples of:
  
      - An item or sequence of items or batch thereof
      - Auxiliary info for the items (i.e. ids, probs).
  
  Raises:
    NotImplementedError: If a non-default argument value is not supported.
    ValueError: If the data spec contains lists that must be converted to
      tuples.
docstr_tok: ["Creates","and","returns","a","dataset","that","returns","entries","from","the","buffer",".","A","single","entry","from","the","dataset","is","the","result","of","the","following","pipeline",":","*","Sample","sequences","from","the","underlying","data","store","*","(","optionally",")","Process","them","with","`","sequence_preprocess_fn","`",",","*","(","optionally",")","Split","them","into","subsequences","of","length","`","num_steps","`","*","(","optionally",")","Batch","them","into","batches","of","size","`","sample_batch_size","`",".","In","practice",",","this","pipeline","is","executed","in","parallel","as","much","as","possible","if","`","num_parallel_calls","!","=","1","`",".","Some","additional","notes",":","If","`","num_steps","is","None","`",",","different","replay","buffers","will","behave","differently",".","For","example",",","`","TFUniformReplayBuffer","`","will","return","single","time","steps","without","a","time","dimension",".","In","contrast",",","e.g.",",","`","EpisodicReplayBuffer","`","will","return","full","sequences","(","since","each","sequence","may","be","an","episode","of","unknown","length",",","the","outermost","shape","dimension","will","be","`","None","`",")",".","If","`","sample_batch_size","is","None","`",",","no","batching","is","performed",";","and","there","is","no","outer","batch","dimension","in","the","returned","Dataset","entries",".","This","setting","is","useful","with","variable","episode","lengths","using","e.g",".","`","EpisodicReplayBuffer","`",",","because","it","allows","the","user","to","get","full","episodes","back",",","and","use","`","tf.data","`","to","build","padded","or","truncated","batches","themselves",".","If","`","single_deterministic_pass","==","True","`",",","the","replay","buffer","will","make","every","attempt","to","ensure","every","time","step","is","visited","once","and","exactly","once","in","a","deterministic","manner","(","though","true","determinism","depends","on","the","underlying","data","store",")",".","Additional","work","may","be","done","to","ensure","minibatches","do","not","have","multiple","rows","from","the","same","episode",".","In","some","cases",",","this","may","mean","arguments","like","`","num_parallel_calls","`","are","ignored",".","Args",":","sample_batch_size",":","(","Optional",".",")","An","optional","batch_size","to","specify","the","number","of","items","to","return",".","If","None","(","default",")",",","a","single","item","is","returned","which","matches","the","data_spec","of","this","class","(","without","a","batch","dimension",")",".","Otherwise",",","a","batch","of","sample_batch_size","items","is","returned",",","where","each","tensor","in","items","will","have","its","first","dimension","equal","to","sample_batch_size","and","the","rest","of","the","dimensions","match","the","corresponding","data_spec",".","num_steps",":","(","Optional",".",")","Optional","way","to","specify","that","sub-episodes","are","desired",".","If","None","(","default",")",",","a","batch","of","single","items","is","returned",".","Otherwise",",","a","batch","of","sub-episodes","is","returned",",","where","a","sub-episode","is","a","sequence","of","consecutive","items","in","the","replay_buffer",".","The","returned","tensors","will","have","first","dimension","equal","to","sample_batch_size","(","if","sample_batch_size","is","not","None",")",",","subsequent","dimension","equal","to","num_steps",",","and","remaining","dimensions","which","match","the","data_spec","of","this","class",".","num_parallel_calls",":","(","Optional",".",")","A","`","tf.int32","`","scalar","`","tf.Tensor","`",",","representing","the","number","elements","to","process","in","parallel",".","If","not","specified",",","elements","will","be","processed","sequentially",".","sequence_preprocess_fn",":","(","Optional",")","fn","for","preprocessing","the","collected","data","before","it","is","split","into","subsequences","of","length","`","num_steps","`",".","Defined","in","`","TFAgent.preprocess_sequence","`",".","Defaults","to","pass","through",".","single_deterministic_pass",":","Python","boolean",".","If","`","True","`",",","the","dataset","will","return","a","single","deterministic","pass","through","its","underlying","data",".","*","*","NOTE","*","*",":","If","the","buffer","is","modified","while","a","Dataset","iterator","is","iterating","over","this","data",",","the","iterator","may","miss","any","new","data","or","otherwise","have","subtly","invalid","data",".","Returns",":","A","dataset","of","type","tf.data.Dataset",",","elements","of","which","are","2-tuples","of",":","-","An","item","or","sequence","of","items","or","batch","thereof","-","Auxiliary","info","for","the","items","(","i.e",".","ids",",","probs",")",".","Raises",":","NotImplementedError",":","If","a","non-default","argument","value","is","not","supported",".","ValueError",":","If","the","data","spec","contains","lists","that","must","be","converted","to","tuples","."]
code: def as_dataset(
      self,
      sample_batch_size=None,
      num_steps=None,
      num_parallel_calls=None,
      sequence_preprocess_fn=None,
      single_deterministic_pass=False,
  ):
      """Creates and returns a dataset that returns entries from the buffer.
  
      A single entry from the dataset is the result of the following pipeline:
  
        * Sample sequences from the underlying data store
        * (optionally) Process them with `sequence_preprocess_fn`,
        * (optionally) Split them into subsequences of length `num_steps`
        * (optionally) Batch them into batches of size `sample_batch_size`.
  
      In practice, this pipeline is executed in parallel as much as possible
      if `num_parallel_calls != 1`.
  
      Some additional notes:
  
      If `num_steps is None`, different replay buffers will behave differently.
      For example, `TFUniformReplayBuffer` will return single time steps without
      a time dimension.  In contrast, e.g., `EpisodicReplayBuffer` will return
      full sequences (since each sequence may be an episode of unknown length,
      the outermost shape dimension will be `None`).
  
      If `sample_batch_size is None`, no batching is performed; and there is no
      outer batch dimension in the returned Dataset entries.  This setting
      is useful with variable episode lengths using e.g. `EpisodicReplayBuffer`,
      because it allows the user to get full episodes back, and use `tf.data`
      to build padded or truncated batches themselves.
  
      If `single_deterministic_pass == True`, the replay buffer will make
      every attempt to ensure every time step is visited once and exactly once
      in a deterministic manner (though true determinism depends on the
      underlying data store).  Additional work may be done to ensure minibatches
      do not have multiple rows from the same episode.  In some cases, this
      may mean arguments like `num_parallel_calls` are ignored.
  
      Args:
        sample_batch_size: (Optional.) An optional batch_size to specify the
          number of items to return. If None (default), a single item is returned
          which matches the data_spec of this class (without a batch dimension).
          Otherwise, a batch of sample_batch_size items is returned, where each
          tensor in items will have its first dimension equal to sample_batch_size
          and the rest of the dimensions match the corresponding data_spec.
        num_steps: (Optional.)  Optional way to specify that sub-episodes are
          desired. If None (default), a batch of single items is returned.
          Otherwise, a batch of sub-episodes is returned, where a sub-episode is a
          sequence of consecutive items in the replay_buffer. The returned tensors
          will have first dimension equal to sample_batch_size (if
          sample_batch_size is not None), subsequent dimension equal to num_steps,
          and remaining dimensions which match the data_spec of this class.
        num_parallel_calls: (Optional.) A `tf.int32` scalar `tf.Tensor`,
          representing the number elements to process in parallel. If not
          specified, elements will be processed sequentially.
        sequence_preprocess_fn: (Optional) fn for preprocessing the collected data
          before it is split into subsequences of length `num_steps`. Defined in
          `TFAgent.preprocess_sequence`.  Defaults to pass through.
        single_deterministic_pass: Python boolean.  If `True`, the dataset will
          return a single deterministic pass through its underlying data.
          **NOTE**: If the buffer is modified while a Dataset iterator is
          iterating over this data, the iterator may miss any new data or
          otherwise have subtly invalid data.
  
      Returns:
        A dataset of type tf.data.Dataset, elements of which are 2-tuples of:
  
          - An item or sequence of items or batch thereof
          - Auxiliary info for the items (i.e. ids, probs).
  
      Raises:
        NotImplementedError: If a non-default argument value is not supported.
        ValueError: If the data spec contains lists that must be converted to
          tuples.
      """
      # data_tf.nest.flatten does not flatten python lists, nest.flatten does.
      if tf.nest.flatten(self._data_spec) != data_nest.flatten(
          self._data_spec
      ):
          raise ValueError(
              "Cannot perform gather; data spec contains lists and this conflicts "
              "with gathering operator.  Convert any lists to tuples.  "
              "For example, if your spec looks like [a, b, c], "
              "change it to (a, b, c).  Spec structure is:\n  {}".format(
                  tf.nest.map_structure(
                      lambda spec: spec.dtype, self._data_spec
                  )
              )
          )
  
      if single_deterministic_pass:
          ds = self._single_deterministic_pass_dataset(
              sample_batch_size=sample_batch_size,
              num_steps=num_steps,
              sequence_preprocess_fn=sequence_preprocess_fn,
              num_parallel_calls=num_parallel_calls,
          )
      else:
          ds = self._as_dataset(
              sample_batch_size=sample_batch_size,
              num_steps=num_steps,
              sequence_preprocess_fn=sequence_preprocess_fn,
              num_parallel_calls=num_parallel_calls,
          )
  
      if self._stateful_dataset:
          options = tf.data.Options()
          if hasattr(options, "experimental_allow_stateful"):
              options.experimental_allow_stateful = True
              ds = ds.with_options(options)
      return ds
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import abc
  import tensorflow
  from tf_agents.utils import common
  from tensorflow.python.data.util import nest
  from tensorflow.python.util import deprecation
label_desc: Use image node_id 8 for calling the ReplayBuffer obj's underlying member method code with example usage: obj.as_dataset(sample_batch_size, num_steps, num_parallel_calls, sequence_preprocess_fn, single_deterministic_pass) and returns: ds