class_docstr: Multi-head attention with Relative Positional embedding.
  https://arxiv.org/pdf/1809.04281.pdf
  
  It learns positional embeddings for a window of neighbours. For keys and values,
  it learns different set of embeddings. Key embeddings are agregated with the attention
  scores and value embeddings are aggregated with the output.
  
  Note:
      Example with relative attention window size 2
      input = [a, b, c, d, e]
      rel_attn_embeddings = [e(t-2), e(t-1), e(t+1), e(t+2)]
  
      So it learns 4 embedding vectors (in total 8) separately for key and value vectors.
  
      Considering the input c
          e(t-2) corresponds to c -> a
          e(t-2) corresponds to c -> b
          e(t-2) corresponds to c -> d
          e(t-2) corresponds to c -> e
  
      These embeddings are shared among different time steps. So input a, b, d and e also uses
      the same embeddings.
  
      Embeddings are ignored when the relative window is out of limit for the first and the last
      n items.
  
  Args:
      channels (int): input and inner layer channels.
      out_channels (int): output channels.
      num_heads (int): number of attention heads.
      rel_attn_window_size (int, optional): relation attention window size.
          If 4, for each time step next and previous 4 time steps are attended.
          If default, relative encoding is disabled and it is a regular transformer.
          Defaults to None.
      heads_share (bool, optional): [description]. Defaults to True.
      dropout_p (float, optional): dropout rate. Defaults to 0..
      input_length (int, optional): intput length for positional encoding. Defaults to None.
      proximal_bias (bool, optional): enable/disable proximal bias as in the paper. Defaults to False.
      proximal_init (bool, optional): enable/disable poximal init as in the paper.
          Init key and query layer weights the same. Defaults to False.
class_docstr_tok: ["Multi-head","attention","with","Relative","Positional","embedding",".","https",":","\/\/arxiv.org\/pdf\/1809.04281.pdf","It","learns","positional","embeddings","for","a","window","of","neighbours",".","For","keys","and","values",",","it","learns","different","set","of","embeddings",".","Key","embeddings","are","agregated","with","the","attention","scores","and","value","embeddings","are","aggregated","with","the","output",".","Note",":","Example","with","relative","attention","window","size","2","input","=","[","a",",","b",",","c",",","d",",","e","]","rel_attn_embeddings","=","[","e","(","t-2",")",",","e","(","t-1",")",",","e","(","t+1",")",",","e","(","t+2",")","]","So","it","learns","4","embedding","vectors","(","in","total","8",")","separately","for","key","and","value","vectors",".","Considering","the","input","c","e","(","t-2",")","corresponds","to","c","-",">","a","e","(","t-2",")","corresponds","to","c","-",">","b","e","(","t-2",")","corresponds","to","c","-",">","d","e","(","t-2",")","corresponds","to","c","-",">","e","These","embeddings","are","shared","among","different","time","steps",".","So","input","a",",","b",",","d","and","e","also","uses","the","same","embeddings",".","Embeddings","are","ignored","when","the","relative","window","is","out","of","limit","for","the","first","and","the","last","n","items",".","Args",":","channels","(","int",")",":","input","and","inner","layer","channels",".","out_channels","(","int",")",":","output","channels",".","num_heads","(","int",")",":","number","of","attention","heads",".","rel_attn_window_size","(","int",",","optional",")",":","relation","attention","window","size",".","If","4",",","for","each","time","step","next","and","previous","4","time","steps","are","attended",".","If","default",",","relative","encoding","is","disabled","and","it","is","a","regular","transformer",".","Defaults","to","None",".","heads_share","(","bool",",","optional",")",":","[","description","]",".","Defaults","to","True",".","dropout_p","(","float",",","optional",")",":","dropout","rate",".","Defaults","to","0","..","input_length","(","int",",","optional",")",":","intput","length","for","positional","encoding",".","Defaults","to","None",".","proximal_bias","(","bool",",","optional",")",":","enable\/disable","proximal","bias","as","in","the","paper",".","Defaults","to","False",".","proximal_init","(","bool",",","optional",")",":","enable\/disable","poximal","init","as","in","the","paper",".","Init","key","and","query","layer","weights","the","same",".","Defaults","to","False","."]
docstr: Convert embedding vestors to a tensor of embeddings
          
docstr_tok: ["Convert","embedding","vestors","to","a","tensor","of","embeddings"]
code: def _get_relative_embeddings(self, relative_embeddings, length):
      """Convert embedding vestors to a tensor of embeddings"""
      # Pad first before slice to avoid using cond ops.
      pad_length = max(length - (self.rel_attn_window_size + 1), 0)
      slice_start_position = max(
          (self.rel_attn_window_size + 1) - length, 0
      )
      slice_end_position = slice_start_position + 2 * length - 1
      if pad_length > 0:
          padded_relative_embeddings = F.pad(
              relative_embeddings, [0, 0, pad_length, pad_length, 0, 0]
          )
      else:
          padded_relative_embeddings = relative_embeddings
      used_relative_embeddings = padded_relative_embeddings[
          :, slice_start_position:slice_end_position
      ]
      return used_relative_embeddings
  
imports: import math
  import torch
  from torch import nn
  from torch.nn import functional
  from TTS.tts.layers.glow_tts.glow import LayerNorm
label_desc: Use image node_id 6 for calling the RelativePositionMultiHeadAttention obj's underlying member method code with example usage: obj._get_relative_embeddings(relative_embeddings, length) and returns: used_relative_embeddings