class_docstr: This class implements the relative multihead implementation similar to that in Transformer XL
  https://arxiv.org/pdf/1901.02860.pdf
  
  Arguments
  ---------
  embed_dim : int
      Size of the encoder feature vectors from which keys and values are computed.
  num_heads: int
      Number of attention heads.
  dropout : float, optional
      Dropout rate.
  vbias: bool, optional
      Whether to use bias for computing value.
  vdim: int, optional
      Size for value. Default is embed_dim (Note each head is embed_dim // num_heads).
  mask_pos_future: bool, optional
      Whether to mask future positional encodings values.
      Must be true for causal applications e.g. decoder.
  Example
  -------
  >>> inputs = torch.rand([6, 60, 512])
  >>> pos_emb = torch.rand([1, 2*60-1, 512])
  >>> net = RelPosMHAXL(num_heads=8, embed_dim=inputs.shape[-1])
  >>> outputs, attn = net(inputs, inputs, inputs, pos_emb)
  >>> outputs.shape
  torch.Size([6, 60, 512])
docstr: Relative shift implementation.
docstr_tok: ["Relative","shift","implementation","."]
code: def rel_shift(self, x):
      """Relative shift implementation."""
      # batch, head, time1, 2*time1-1.
  
      b, h, qlen, pos_len = x.size()  # (b, h, t1, t2)
      # need to add a column of zeros on the left side of last dimension to perform the relative shifting
      x = torch.nn.functional.pad(x, pad=(1, 0))  # (b, h, t1, t2+1)
      x = x.view(b, h, -1, qlen)  # (b, h, t2+1, t1)
      # need to drop the first row
      x = x[:, :, 1:].view(b, h, qlen, pos_len)  # (b, h, t1, t2)
  
      if self.mask_pos_future:
          ones = torch.ones((x.size(2), x.size(3)), device=x.device)
          x = (
              x
              * torch.tril(ones, x.size(3) - x.size(2))[
                  None, None, :, :
              ]
          )
  
      return x[..., : pos_len // 2 + 1]
  
imports: import torch
  import logging
  import torch.nn
  import numpy
  from typing import Optional
  from speechbrain.dataio.dataio import length_to_mask
  import torch.nn.functional
  import math
label_desc: Use image node_id 3 for calling the RelPosMHAXL obj's underlying member method code with example usage: obj.rel_shift(x) and returns: x