class_docstr: Implements a Transformer Encoder Layer used in BERT/XLM style pre-trained
  models.
docstr: LayerNorm is applied either before or after the self-attention/ffn
  modules similar to the original Transformer imlementation.
docstr_tok: ["LayerNorm","is","applied","either","before","or","after","the","self-attention\/ffn","modules","similar","to","the","original","Transformer","imlementation","."]
code: def forward(
      self,
      x: torch.Tensor,
      self_attn_mask: torch.Tensor = None,
      self_attn_padding_mask: torch.Tensor = None,
      need_weights: bool = False,
      pos_bias=None,
  ):
      """
      LayerNorm is applied either before or after the self-attention/ffn
      modules similar to the original Transformer imlementation.
      """
      residual = x
  
      if self.layer_norm_first:
          x = self.self_attn_layer_norm(x)
          x, attn, pos_bias = self.self_attn(
              query=x,
              key=x,
              value=x,
              key_padding_mask=self_attn_padding_mask,
              need_weights=False,
              attn_mask=self_attn_mask,
              position_bias=pos_bias,
          )
          x = self.dropout1(x)
          x = residual + x
  
          residual = x
          x = self.final_layer_norm(x)
          if self.activation_name == "glu":
              x = self.fc1(x)
          else:
              x = self.activation_fn(self.fc1(x))
          x = self.dropout2(x)
          x = self.fc2(x)
          x = self.dropout3(x)
          x = residual + x
      else:
          x, attn, pos_bias = self.self_attn(
              query=x,
              key=x,
              value=x,
              key_padding_mask=self_attn_padding_mask,
              need_weights=need_weights,
              attn_mask=self_attn_mask,
              position_bias=pos_bias,
          )
  
          x = self.dropout1(x)
          x = residual + x
  
          x = self.self_attn_layer_norm(x)
  
          residual = x
          if self.activation_name == "glu":
              x = self.fc1(x)
          else:
              x = self.activation_fn(self.fc1(x))
          x = self.dropout2(x)
          x = self.fc2(x)
          x = self.dropout3(x)
          x = residual + x
          x = self.final_layer_norm(x)
  
      return x, attn, pos_bias
  
imports: import logging
  import math
  from typing import List, Optional, Tuple
  import numpy
  import torch
  import torch.nn
  import torch.nn.functional
  from torch.nn import LayerNorm
  from TTS.vc.modules.freevc.wavlm.modules import Fp32GroupNorm, Fp32LayerNorm, GLU_Linear, GradMultiply, MultiheadAttention, SamePad, TransposeLast, get_activation_fn, init_bert_params
label_desc: Use image node_id 2 for calling the TransformerSentenceEncoderLayer obj's underlying member method code with example usage: obj.forward(x, _attn_mask, _attn_padding_mask, need_weights, pos_bias) and returns: x,  attn,  pos_bias