docstr: An implementation of positional encoding as described in 'Attention is All you Need' by Vaswani et al. (2017)
  
  Parameters
  ----------
  d_model
      The number of expected features in the transformer encoder/decoder inputs.
      Last dimension of the input.
  dropout
      Fraction of neurons affected by Dropout (default=0.1).
  max_len
      The dimensionality of the computed positional encoding array.
      Only its first "input_size" elements will be considered in the output.
  
  Inputs
  ------
  x of shape `(batch_size, input_size, d_model)`
      Tensor containing the embedded time series.
  
  Outputs
  -------
  y of shape `(batch_size, input_size, d_model)`
      Tensor containing the embedded time series enhanced with positional encoding.
docstr_tok: ["An","implementation","of","positional","encoding","as","described","in","'Attention","is","All","you","Need","'","by","Vaswani","et","al",".","(","2017",")","Parameters","--","--","--","--","--","d_model","The","number","of","expected","features","in","the","transformer","encoder\/decoder","inputs",".","Last","dimension","of","the","input",".","dropout","Fraction","of","neurons","affected","by","Dropout","(","default=0.1",")",".","max_len","The","dimensionality","of","the","computed","positional","encoding","array",".","Only","its","first","``","input_size","''","elements","will","be","considered","in","the","output",".","Inputs","--","--","--","x","of","shape","`","(","batch_size",",","input_size",",","d_model",")","`","Tensor","containing","the","embedded","time","series",".","Outputs","--","--","--","-","y","of","shape","`","(","batch_size",",","input_size",",","d_model",")","`","Tensor","containing","the","embedded","time","series","enhanced","with","positional","encoding","."]
code: def __init__(self, d_model, dropout=0.1, max_len=500):
      """An implementation of positional encoding as described in 'Attention is All you Need' by Vaswani et al. (2017)
  
      Parameters
      ----------
      d_model
          The number of expected features in the transformer encoder/decoder inputs.
          Last dimension of the input.
      dropout
          Fraction of neurons affected by Dropout (default=0.1).
      max_len
          The dimensionality of the computed positional encoding array.
          Only its first "input_size" elements will be considered in the output.
  
      Inputs
      ------
      x of shape `(batch_size, input_size, d_model)`
          Tensor containing the embedded time series.
  
      Outputs
      -------
      y of shape `(batch_size, input_size, d_model)`
          Tensor containing the embedded time series enhanced with positional encoding.
      """
      super().__init__()
      self.dropout = nn.Dropout(p=dropout)
  
      pe = torch.zeros(max_len, d_model)
      position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(
          1
      )
      div_term = torch.exp(
          torch.arange(0, d_model, 2).float()
          * (-math.log(10000.0) / d_model)
      )
      pe[:, 0::2] = torch.sin(position * div_term)
      pe[:, 1::2] = torch.cos(position * div_term)
      pe = pe.unsqueeze(0).transpose(0, 1)
      self.register_buffer("pe", pe)
  
imports: import math
  from typing import Optional, Tuple, Union
  import torch
  import torch.nn
  from darts.logging import get_logger, raise_if, raise_if_not, raise_log
  from darts.models.components import glu_variants, layer_norm_variants
  from darts.models.components.glu_variants import GLU_FFN
  from darts.models.components.transformer import CustomFeedForwardDecoderLayer, CustomFeedForwardEncoderLayer
  from darts.models.forecasting.pl_forecasting_module import PLPastCovariatesModule, io_processor
  from darts.models.forecasting.torch_forecasting_model import PastCovariatesTorchModel
label_desc: Use image node_id 1 to create a new _PositionalEncoding object from inherited base classes: nn with example: obj = _PositionalEncoding(d_model, dropout, max_len)