class_docstr: A memory module that caches features for recent tokens.
  
  When the number of tokens cached is equal to the chunk size, this is
  equivalent to the memory used by Transformer-XL
  (https://arxiv.org/abs/1901.02860)
class_docstr_tok: ["A","memory","module","that","caches","features","for","recent","tokens",".","When","the","number","of","tokens","cached","is","equal","to","the","chunk","size",",","this","is","equivalent","to","the","memory","used","by","Transformer-XL","(","https",":","\/\/arxiv.org\/abs\/1901.02860",")"]
code: def __init__(self, name, hparams):
      hidden_size = hparams.hidden_size
      self.chunk_length = hparams.split_targets_chunk_length
      assert (
          self.chunk_length > 0
      ), "Chunking is required to use recurrent memory"
  
      if (
          hasattr(hparams, "num_memory_items")
          and hparams.num_memory_items > 0
      ):
          self.tokens_to_cache = hparams.num_memory_items
      else:
          self.tokens_to_cache = self.chunk_length
  
      # TODO(kitaev): The implementation of the chunking code makes it somewhat
      # convoluted to figure out how many actual sequences we can have per batch.
      # The data pipeline should be revisited at some point.
      if (
          hasattr(hparams, "recurrent_memory_batch_size")
          and hparams.recurrent_memory_batch_size > 0
      ):
          batch_size_in_sequences = hparams.recurrent_memory_batch_size
      else:
          batch_size_in_sequences = (
              hparams.batch_size / hparams.max_length
          )
  
      memory_shape = [
          batch_size_in_sequences,
          self.tokens_to_cache,
          hidden_size,
      ]
      bias_shape = [batch_size_in_sequences, 1, 1, self.tokens_to_cache]
  
      with tf.variable_scope(name):
          self.previous_segment = tf.get_variable(
              "memsegment",
              (batch_size_in_sequences,),
              dtype=tf.int32,
              trainable=False,
              collections=[tf.GraphKeys.LOCAL_VARIABLES],
              initializer=tf.constant_initializer(0),
          )
  
          self.previous_vals = tf.get_variable(
              "memvals",
              memory_shape,
              dtype=tf.float32,
              trainable=False,
              collections=[tf.GraphKeys.LOCAL_VARIABLES],
              initializer=tf.constant_initializer(0.0),
          )
  
          self.previous_bias = tf.get_variable(
              "membias",
              bias_shape,
              dtype=tf.float32,
              trainable=False,
              collections=[tf.GraphKeys.LOCAL_VARIABLES],
              initializer=tf.constant_initializer(-1e9),
          )
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  from tensor2tensor.layers import common_layers
  import tensorflow.compat.v1
label_desc: Use image node_id 1 to create a new RecentTokensMemory object from inherited base classes: RecurrentMemory with example: obj = RecentTokensMemory(name, hparams)