class_docstr: Sinusoid position embedder that maps position indexes into embeddings
  via sinusoid calculation. This module does not have trainable parameters.
  Used in, e.g., Transformer models
  `(Vaswani et al.) "Attention Is All You Need"`.
  
  Each channel of the input Tensor is incremented by a sinusoid of a
  different frequency and phase.
  This allows attention to learn to use absolute and relative positions.
  
  Timing signals should be added to some precursors of both the query
  and the memory inputs to attention.
  The use of relative position is possible because `sin(x+y)` and
  `cos(x+y)` can be expressed in terms of `y`, `sin(x)`, and `cos(x)`.
  In particular, we use a geometric sequence of timescales starting with
  min_timescale and ending with max_timescale.  The number of different
  timescales is equal to ``dim / 2``. For each timescale, we
  generate the two sinusoidal signals `sin(timestep/timescale)` and
  `cos(timestep/timescale)`.  All of these sinusoids are concatenated in
  the dim dimension.
  
  Args:
      position_size (int): The number of possible positions, e.g., the maximum
          sequence length. Set ``position_size=None`` and
          ``hparams['cache_embeddings']=False`` to enable infinite large or
          negative position indexes.
  
  .. document private functions
  .. automethod:: _build
class_docstr_tok: ["Sinusoid","position","embedder","that","maps","position","indexes","into","embeddings","via","sinusoid","calculation",".","This","module","does","not","have","trainable","parameters",".","Used","in",",","e.g.",",","Transformer","models","`","(","Vaswani","et","al",".",")","``","Attention","Is","All","You","Need","''","`",".","Each","channel","of","the","input","Tensor","is","incremented","by","a","sinusoid","of","a","different","frequency","and","phase",".","This","allows","attention","to","learn","to","use","absolute","and","relative","positions",".","Timing","signals","should","be","added","to","some","precursors","of","both","the","query","and","the","memory","inputs","to","attention",".","The","use","of","relative","position","is","possible","because","`","sin","(","x+y",")","`","and","`","cos","(","x+y",")","`","can","be","expressed","in","terms","of","`","y","`",",","`","sin","(","x",")","`",",","and","`","cos","(","x",")","`",".","In","particular",",","we","use","a","geometric","sequence","of","timescales","starting","with","min_timescale","and","ending","with","max_timescale",".","The","number","of","different","timescales","is","equal","to","``","dim","\/","2","``",".","For","each","timescale",",","we","generate","the","two","sinusoidal","signals","`","sin","(","timestep\/timescale",")","`","and","`","cos","(","timestep\/timescale",")","`",".","All","of","these","sinusoids","are","concatenated","in","the","dim","dimension",".","Args",":","position_size","(","int",")",":","The","number","of","possible","positions",",","e.g.",",","the","maximum","sequence","length",".","Set","``","position_size=None","``","and","``","hparams","[","'cache_embeddings","'","]","=False","``","to","enable","infinite","large","or","negative","position","indexes",".","..","document","private","functions","..","automethod",":",":","_build"]
code: def __init__(self, position_size, hparams=None):
      EmbedderBase.__init__(self, hparams=hparams)
  
      self._num_embeds = position_size
      self._dim = self._hparams.dim
      self._cache_embeddings = self._hparams.cache_embeddings
  
      num_timescales = self._dim // 2
      min_timescale = self._hparams.min_timescale
      max_timescale = self._hparams.max_timescale
  
      log_timescale_increment = math.log(
          float(max_timescale) / float(min_timescale)
      ) / (tf.cast(num_timescales, tf.float32) - 1)
      num_range = tf.range(num_timescales, dtype=tf.float32)
      inv_timescales = min_timescale * tf.exp(
          num_range * -log_timescale_increment
      )
      self.inv_timescales = inv_timescales
  
      if self._cache_embeddings:
          if position_size is None:
              raise ValueError(
                  "'position_size' must not be None when "
                  "'cache_embeddings' is set to True"
              )
          positions = tf.range(position_size, dtype=tf.float32)
          signal = self._compute_embeddings(positions)
          self.signal = signal
  
imports: import math
  import tensorflow
  from texar.tf.modules.embedders.embedder_base import EmbedderBase
  from texar.tf.modules.embedders import embedder_utils
  from texar.tf.utils.mode import is_train_mode
  from texar.tf.utils.shapes import mask_sequences
  from texar.tf.utils.shapes import shape_list
label_desc: Use image node_id 1 to create a new SinusoidsPositionEmbedder object from inherited base classes: EmbedderBase with example: obj = SinusoidsPositionEmbedder(position_size, hparams)