docstr: Use a geometric sequence of timescales.
  
  It is calculated as
  [sin(wi x), cos(wi x), sin(wi x), cos(wi x), ...]
  
  By default, we initialize wi to be (1 / 10000) ^ (1 / (units//2 - 1))
  
  
  Parameters
  ----------
  units
      The number of units for positional embedding
  learnable
      Whether to make the Sinusoidal positional embedding learnable.
      If it is turned on, we will also update the frequency of this layer.
      See "[ICLR2021] On Position Embeddings in BERT" for more detail.
docstr_tok: ["Use","a","geometric","sequence","of","timescales",".","It","is","calculated","as","[","sin","(","wi","x",")",",","cos","(","wi","x",")",",","sin","(","wi","x",")",",","cos","(","wi","x",")",",","...","]","By","default",",","we","initialize","wi","to","be","(","1","\/","10000",")","^","(","1","\/","(","units\/\/2","-","1",")",")","Parameters","--","--","--","--","--","units","The","number","of","units","for","positional","embedding","learnable","Whether","to","make","the","Sinusoidal","positional","embedding","learnable",".","If","it","is","turned","on",",","we","will","also","update","the","frequency","of","this","layer",".","See","``","[","ICLR2021","]","On","Position","Embeddings","in","BERT","''","for","more","detail","."]
code: def __init__(self, units: int, learnable=False):
      """Use a geometric sequence of timescales.
  
      It is calculated as
      [sin(wi x), cos(wi x), sin(wi x), cos(wi x), ...]
  
      By default, we initialize wi to be (1 / 10000) ^ (1 / (units//2 - 1))
  
  
      Parameters
      ----------
      units
          The number of units for positional embedding
      learnable
          Whether to make the Sinusoidal positional embedding learnable.
          If it is turned on, we will also update the frequency of this layer.
          See "[ICLR2021] On Position Embeddings in BERT" for more detail.
  
      """
      super().__init__()
  
      def _init_sinusoidal_base(units):
          half_units = units // 2
          val = np.log(10000) / (half_units - 1)
          val = np.exp(np.arange(half_units, dtype=np.float32) * -val)
          return val
  
      default_sinusoidal_base = _init_sinusoidal_base(units)
  
      self.freq = nn.Parameter(
          data=th.tensor(default_sinusoidal_base),
          requires_grad=learnable,
      )
      self._units = units
      self._learnable = learnable
  
imports: import math
  from collections import OrderedDict
  import torch
  import numpy
  from torch import nn
  from typing import Union
  from .utils import to_torch_dtype
label_desc: Use image node_id 1 to create a new SinusoidalPositionalEmbedding object from inherited base classes: nn with example: obj = SinusoidalPositionalEmbedding(units, learnable)