class_docstr: Reverb ReplayBuffer exposed as a TF-Agents replay buffer.
class_docstr_tok: ["Reverb","ReplayBuffer","exposed","as","a","TF-Agents","replay","buffer","."]
docstr: Initializes a reverb replay buffer.
  
  *NOTE*: If the user calls `as_dataset()` with a value of `num_steps` that
  does not match the `sequence_length` used in the observer (or if
  `sequence_length is None` varies from episode to episode), then individual
  sequences will be truncated to the highest possible multiple of `num_steps`
  before being split. In this case, **data loss may occur**. For example, in
  sparse reward cases where there's a reward only at the final frame and the
  sequence length is not a multiple of `num_steps`, the reward will be
  dropped.
  
  When using sparse rewards at the end of sequences, the user should prefer
  to either request full episodes (e.g., call `as_dataset` with
  `num_steps=None` and `sample_batch_size=None` (for variable-length
  episodes)); or use `sequence_preprocess_fn` to first propagate the rewards
  from the end of the sequence through to the earlier frames using e.g.
  temporal discounting.
  
  Args:
    data_spec: Spec for the data held in the replay buffer.
    table_name: Name of the table that will be sampled.
    sequence_length: (can be set to `None`, i.e unknown) The number of
      timesteps that each sample consists of. If not `None`, then the lengths
      of samples received from the server will be validated against this
      number.  **NOTE** This replay buffer will be at its most performant if
      the `sequence_length` here is equal to `num_steps` passed to
      `as_dataset`, and is also used when writing to the replay buffer (for
      example, see the `sequence_lengths` argument of the `Reverb.*Observer`
      classes).
    server_address: (Optional) Address of the reverb replay server. One of
      `server_address` or `local_server` must be provided.
    local_server: (Optional) An instance of `reverb.Server` that holds the
      replay's data.
    dataset_buffer_size: (Optional) This is the prefetch buffer size (in
      number of items) of the Reverb Dataset object.  A good rule of thumb is
      to set this value to 2-3x times the sample_batch_size you will use.
    max_cycle_length: (Optional) The number of sequences used to populate the
      batches of `as_dataset`.  By default, `min(32, sample_batch_size)` is
      used, but the number can be between `1` and `sample_batch_size`.
    num_workers_per_iterator: (Defaults to -1, i.e auto selected) The number
      of worker threads to create per dataset iterator. When the selected
      table uses a FIFO or Heap sampler (i.e a queue) then exactly 1 worker
      must be used to avoid races causing invalid ordering of items. For all
      other samplers, this value should be roughly equal to the number of
      threads available on the CPU.
    max_samples_per_stream: (Defaults to -1, i.e auto selected) The maximum
      number of samples to fetch from a stream before a new call is made.
      Keeping this number low ensures that the data is fetched uniformly from
      all servers.
    rate_limiter_timeout_ms: (Defaults to -1: infinite).  Timeout (in
      milliseconds) to wait on the rate limiter when sampling from the table.
      If `rate_limiter_timeout_ms >= 0`, this is the timeout passed to
      `Table::Sample` describing how long to wait for the rate limiter to
      allow sampling.
docstr_tok: ["Initializes","a","reverb","replay","buffer",".","*","NOTE","*",":","If","the","user","calls","`","as_dataset","(",")","`","with","a","value","of","`","num_steps","`","that","does","not","match","the","`","sequence_length","`","used","in","the","observer","(","or","if","`","sequence_length","is","None","`","varies","from","episode","to","episode",")",",","then","individual","sequences","will","be","truncated","to","the","highest","possible","multiple","of","`","num_steps","`","before","being","split",".","In","this","case",",","*","*","data","loss","may","occur","*","*",".","For","example",",","in","sparse","reward","cases","where","there","'s","a","reward","only","at","the","final","frame","and","the","sequence","length","is","not","a","multiple","of","`","num_steps","`",",","the","reward","will","be","dropped",".","When","using","sparse","rewards","at","the","end","of","sequences",",","the","user","should","prefer","to","either","request","full","episodes","(","e.g.",",","call","`","as_dataset","`","with","`","num_steps=None","`","and","`","sample_batch_size=None","`","(","for","variable-length","episodes",")",")",";","or","use","`","sequence_preprocess_fn","`","to","first","propagate","the","rewards","from","the","end","of","the","sequence","through","to","the","earlier","frames","using","e.g",".","temporal","discounting",".","Args",":","data_spec",":","Spec","for","the","data","held","in","the","replay","buffer",".","table_name",":","Name","of","the","table","that","will","be","sampled",".","sequence_length",":","(","can","be","set","to","`","None","`",",","i.e","unknown",")","The","number","of","timesteps","that","each","sample","consists","of",".","If","not","`","None","`",",","then","the","lengths","of","samples","received","from","the","server","will","be","validated","against","this","number",".","*","*","NOTE","*","*","This","replay","buffer","will","be","at","its","most","performant","if","the","`","sequence_length","`","here","is","equal","to","`","num_steps","`","passed","to","`","as_dataset","`",",","and","is","also","used","when","writing","to","the","replay","buffer","(","for","example",",","see","the","`","sequence_lengths","`","argument","of","the","`","Reverb",".","*","Observer","`","classes",")",".","server_address",":","(","Optional",")","Address","of","the","reverb","replay","server",".","One","of","`","server_address","`","or","`","local_server","`","must","be","provided",".","local_server",":","(","Optional",")","An","instance","of","`","reverb.Server","`","that","holds","the","replay","'s","data",".","dataset_buffer_size",":","(","Optional",")","This","is","the","prefetch","buffer","size","(","in","number","of","items",")","of","the","Reverb","Dataset","object",".","A","good","rule","of","thumb","is","to","set","this","value","to","2-3x","times","the","sample_batch_size","you","will","use",".","max_cycle_length",":","(","Optional",")","The","number","of","sequences","used","to","populate","the","batches","of","`","as_dataset","`",".","By","default",",","`","min","(","32",",","sample_batch_size",")","`","is","used",",","but","the","number","can","be","between","`","1","`","and","`","sample_batch_size","`",".","num_workers_per_iterator",":","(","Defaults","to","-1",",","i.e","auto","selected",")","The","number","of","worker","threads","to","create","per","dataset","iterator",".","When","the","selected","table","uses","a","FIFO","or","Heap","sampler","(","i.e","a","queue",")","then","exactly","1","worker","must","be","used","to","avoid","races","causing","invalid","ordering","of","items",".","For","all","other","samplers",",","this","value","should","be","roughly","equal","to","the","number","of","threads","available","on","the","CPU",".","max_samples_per_stream",":","(","Defaults","to","-1",",","i.e","auto","selected",")","The","maximum","number","of","samples","to","fetch","from","a","stream","before","a","new","call","is","made",".","Keeping","this","number","low","ensures","that","the","data","is","fetched","uniformly","from","all","servers",".","rate_limiter_timeout_ms",":","(","Defaults","to","-1",":","infinite",")",".","Timeout","(","in","milliseconds",")","to","wait","on","the","rate","limiter","when","sampling","from","the","table",".","If","`","rate_limiter_timeout_ms",">","=","0","`",",","this","is","the","timeout","passed","to","`","Table",":",":Sample","`","describing","how","long","to","wait","for","the","rate","limiter","to","allow","sampling","."]
code: def __init__(
      self,
      data_spec,
      table_name,
      sequence_length,
      server_address=None,
      local_server=None,
      dataset_buffer_size=None,
      max_cycle_length=32,
      num_workers_per_iterator=-1,
      max_samples_per_stream=-1,
      rate_limiter_timeout_ms=-1,
  ):
      """Initializes a reverb replay buffer.
  
      *NOTE*: If the user calls `as_dataset()` with a value of `num_steps` that
      does not match the `sequence_length` used in the observer (or if
      `sequence_length is None` varies from episode to episode), then individual
      sequences will be truncated to the highest possible multiple of `num_steps`
      before being split. In this case, **data loss may occur**. For example, in
      sparse reward cases where there's a reward only at the final frame and the
      sequence length is not a multiple of `num_steps`, the reward will be
      dropped.
  
      When using sparse rewards at the end of sequences, the user should prefer
      to either request full episodes (e.g., call `as_dataset` with
      `num_steps=None` and `sample_batch_size=None` (for variable-length
      episodes)); or use `sequence_preprocess_fn` to first propagate the rewards
      from the end of the sequence through to the earlier frames using e.g.
      temporal discounting.
  
      Args:
        data_spec: Spec for the data held in the replay buffer.
        table_name: Name of the table that will be sampled.
        sequence_length: (can be set to `None`, i.e unknown) The number of
          timesteps that each sample consists of. If not `None`, then the lengths
          of samples received from the server will be validated against this
          number.  **NOTE** This replay buffer will be at its most performant if
          the `sequence_length` here is equal to `num_steps` passed to
          `as_dataset`, and is also used when writing to the replay buffer (for
          example, see the `sequence_lengths` argument of the `Reverb.*Observer`
          classes).
        server_address: (Optional) Address of the reverb replay server. One of
          `server_address` or `local_server` must be provided.
        local_server: (Optional) An instance of `reverb.Server` that holds the
          replay's data.
        dataset_buffer_size: (Optional) This is the prefetch buffer size (in
          number of items) of the Reverb Dataset object.  A good rule of thumb is
          to set this value to 2-3x times the sample_batch_size you will use.
        max_cycle_length: (Optional) The number of sequences used to populate the
          batches of `as_dataset`.  By default, `min(32, sample_batch_size)` is
          used, but the number can be between `1` and `sample_batch_size`.
        num_workers_per_iterator: (Defaults to -1, i.e auto selected) The number
          of worker threads to create per dataset iterator. When the selected
          table uses a FIFO or Heap sampler (i.e a queue) then exactly 1 worker
          must be used to avoid races causing invalid ordering of items. For all
          other samplers, this value should be roughly equal to the number of
          threads available on the CPU.
        max_samples_per_stream: (Defaults to -1, i.e auto selected) The maximum
          number of samples to fetch from a stream before a new call is made.
          Keeping this number low ensures that the data is fetched uniformly from
          all servers.
        rate_limiter_timeout_ms: (Defaults to -1: infinite).  Timeout (in
          milliseconds) to wait on the rate limiter when sampling from the table.
          If `rate_limiter_timeout_ms >= 0`, this is the timeout passed to
          `Table::Sample` describing how long to wait for the rate limiter to
          allow sampling.
      """
      if (server_address is None) == (local_server is None):
          raise ValueError(
              "Exactly one of the server_address or local_server must be provided."
          )
  
      self._table_name = table_name
      self._sequence_length = sequence_length
      self._local_server = local_server
      self._server_address = server_address
      self._dataset_buffer_size = dataset_buffer_size
      self._max_cycle_length = max_cycle_length
      # TODO(b/156531956) Remove these
      self._num_workers_per_iterator = num_workers_per_iterator
      self._max_samples_per_stream = max_samples_per_stream
      self._rate_limiter_timeout_ms = rate_limiter_timeout_ms
  
      if local_server:
          self._server_address = "localhost:{}".format(
              local_server.port
          )
  
      self._py_client = reverb.Client(self._server_address)
      self._tf_client = reverb.TFClient(
          self._server_address, "rb_tf_client"
      )
      self._table_info = self.get_table_info()
      sampler = self._table_info.sampler_options
      remover = self._table_info.remover_options
      self._deterministic_table = (
          sampler.is_deterministic and remover.is_deterministic
      )
  
      capacity = self._table_info.max_size
      super(ReverbReplayBuffer, self).__init__(
          data_spec=data_spec, capacity=capacity, stateful_dataset=True
      )
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import gin
  import tensorflow.compat.v2
  from tf_agents.replay_buffers import replay_buffer
  from tf_agents.typing import types
  from tf_agents.utils import lazy_loader
label_desc: Use image node_id 1 to create a new ReverbReplayBuffer object from inherited base classes: replay_buffer with example: obj = ReverbReplayBuffer(data_spec, table_name, sequence_length, server_address, local_server, dataset_buffer_size, max_cycle_length, num_workers_per_iterator, max_samples_per_stream, rate_limiter_timeout_ms)