class_docstr: A stack of layers for Universal Transformer.
  
  This implementation is largely adapted from t2t universal transformer
  implementation. Reference:
  third_party/py/tensor2tensor/models/research
class_docstr_tok: ["A","stack","of","layers","for","Universal","Transformer",".","This","implementation","is","largely","adapted","from","t2t","universal","transformer","implementation",".","Reference",":","third_party\/py\/tensor2tensor\/models\/research"]
docstr: Gets a bunch of sinusoids of different frequencies.
  
  Each channel of the input Tensor is incremented by a sinusoid of a different
  frequency and phase.
  
  This allows attention to learn to use absolute and relative positions.
  Timing signals should be added to some precursors of both the query and the
  memory inputs to attention.
  
  The use of relative position is possible because sin(x+y) and cos(x+y) can
  be expressed in terms of y, sin(x) and cos(x).
  
  In particular, we use a geometric sequence of timescales starting with
  min_timescale and ending with max_timescale.  The number of different
  timescales is equal to channels / 2. For each timescale, we
  generate the two sinusoidal signals sin(timestep/timescale) and
  cos(timestep/timescale).  All of these sinusoids are concatenated in
  the channels dimension.
  
  Args:
    context: mtf context.
    length: a mtf.Dimension, length of timing signal sequence.
    channels: a mtf.Dimension, size of timing embeddings to create.
    The number of different timescales is equal to channels / 2.
    min_timescale: a float
    max_timescale: a float
    start_index: index of first position
  
  Returns:
    a Tensor of timing signals [1, length, channels]
docstr_tok: ["Gets","a","bunch","of","sinusoids","of","different","frequencies",".","Each","channel","of","the","input","Tensor","is","incremented","by","a","sinusoid","of","a","different","frequency","and","phase",".","This","allows","attention","to","learn","to","use","absolute","and","relative","positions",".","Timing","signals","should","be","added","to","some","precursors","of","both","the","query","and","the","memory","inputs","to","attention",".","The","use","of","relative","position","is","possible","because","sin","(","x+y",")","and","cos","(","x+y",")","can","be","expressed","in","terms","of","y",",","sin","(","x",")","and","cos","(","x",")",".","In","particular",",","we","use","a","geometric","sequence","of","timescales","starting","with","min_timescale","and","ending","with","max_timescale",".","The","number","of","different","timescales","is","equal","to","channels","\/","2",".","For","each","timescale",",","we","generate","the","two","sinusoidal","signals","sin","(","timestep\/timescale",")","and","cos","(","timestep\/timescale",")",".","All","of","these","sinusoids","are","concatenated","in","the","channels","dimension",".","Args",":","context",":","mtf","context",".","length",":","a","mtf.Dimension",",","length","of","timing","signal","sequence",".","channels",":","a","mtf.Dimension",",","size","of","timing","embeddings","to","create",".","The","number","of","different","timescales","is","equal","to","channels","\/","2.","min_timescale",":","a","float","max_timescale",":","a","float","start_index",":","index","of","first","position","Returns",":","a","Tensor","of","timing","signals","[","1",",","length",",","channels","]"]
code: def get_timing_signal_1d(
      self,
      context,
      length,
      channels,
      min_timescale=1.0,
      max_timescale=1.0e4,
      start_index=0,
  ):
      """Gets a bunch of sinusoids of different frequencies.
  
      Each channel of the input Tensor is incremented by a sinusoid of a different
      frequency and phase.
  
      This allows attention to learn to use absolute and relative positions.
      Timing signals should be added to some precursors of both the query and the
      memory inputs to attention.
  
      The use of relative position is possible because sin(x+y) and cos(x+y) can
      be expressed in terms of y, sin(x) and cos(x).
  
      In particular, we use a geometric sequence of timescales starting with
      min_timescale and ending with max_timescale.  The number of different
      timescales is equal to channels / 2. For each timescale, we
      generate the two sinusoidal signals sin(timestep/timescale) and
      cos(timestep/timescale).  All of these sinusoids are concatenated in
      the channels dimension.
  
      Args:
        context: mtf context.
        length: a mtf.Dimension, length of timing signal sequence.
        channels: a mtf.Dimension, size of timing embeddings to create.
        The number of different timescales is equal to channels / 2.
        min_timescale: a float
        max_timescale: a float
        start_index: index of first position
  
      Returns:
        a Tensor of timing signals [1, length, channels]
      """
  
      position = context.get_position() + start_index
      num_timescales = mtf.constant(context.mesh, channels.size // 2)
      log_timescale_increment = math.log(
          float(max_timescale) / float(min_timescale)
      ) / mtf.maximum(num_timescales - 1, 1)
      channel_dim_name = channels.name
      inv_timescales = min_timescale * mtf.exp(
          mtf.mtf_range(
              context.mesh,
              mtf.Dimension(channel_dim_name, channels.size // 2),
              context.activation_dtype,
          )
          * -log_timescale_increment
      )
  
      scaled_time = position * inv_timescales
      # Please note that this slightly differs from the published paper.
      # See a discussion here:
      # https://github.com/tensorflow/tensor2tensor/pull/177
      #    concat_dim_name = scaled_time.shape.dimension_names[1]
      concat_dim_name = channels.name
      signal = mtf.concat(
          [mtf.sin(scaled_time), mtf.cos(scaled_time)],
          concat_dim_name=concat_dim_name,
      )
  
      if channels.size % 2 != 0:
          raise NotImplementedError("Odd channel size not implemented.")
      new_dims = (
          [mtf.Dimension("expanded", 1)]
          + length.shape.dims
          + channels.shape.dim
      )
      signal = mtf.reshape(signal, mtf.Shape(new_dims))
      return signal
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import math
  import gin
  import mesh_tensorflow
  from mesh_tensorflow.transformer import transformer
  import tensorflow.compat.v1
label_desc: Use image node_id 2 for calling the UTLayerStack obj's underlying member method code with example usage: obj.get_timing_signal_1d(context, length, channels, min_timescale, max_timescale, start_index) and returns: signal