class_docstr: This is an implementation of Conformer encoder layer.
  
  Arguments
  ----------
  d_model : int
      The expected size of the input embedding.
  d_ffn : int
      Hidden size of self-attention Feed Forward layer.
  nhead : int
      Number of attention heads.
  kernel_size : int, optional
      Kernel size of convolution model.
  kdim : int, optional
      Dimension of the key.
  vdim : int, optional
      Dimension of the value.
  activation: torch.nn.Module
       Activation function used in each Conformer layer.
  bias : bool, optional
      Whether  convolution module.
  dropout : int, optional
      Dropout for the encoder.
  causal: bool, optional
      Whether the convolutions should be causal or not.
  attention_type: str, optional
      type of attention layer, e.g. regulaMHA for regular MultiHeadAttention.
  
  Example
  -------
  >>> import torch
  >>> x = torch.rand((8, 60, 512))
  >>> pos_embs = torch.rand((1, 2*60-1, 512))
  >>> net = ConformerEncoderLayer(d_ffn=512, nhead=8, d_model=512, kernel_size=3)
  >>> output = net(x, pos_embs=pos_embs)
  >>> output[0].shape
  torch.Size([8, 60, 512])
code: def __init__(
      self,
      d_model,
      d_ffn,
      nhead,
      kernel_size=31,
      kdim=None,
      vdim=None,
      activation=Swish,
      bias=True,
      dropout=0.0,
      causal=False,
      attention_type="RelPosMHAXL",
  ):
      super().__init__()
  
      if attention_type == "regularMHA":
          self.mha_layer = MultiheadAttention(
              nhead=nhead,
              d_model=d_model,
              dropout=dropout,
              kdim=kdim,
              vdim=vdim,
          )
      elif attention_type == "RelPosMHAXL":
          # transformerXL style positional encoding
          self.mha_layer = RelPosMHAXL(
              num_heads=nhead,
              embed_dim=d_model,
              dropout=dropout,
              mask_pos_future=causal,
          )
      elif attention_type == "hypermixing":
          self.mha_layer = HyperMixing(
              input_output_dim=d_model,
              hypernet_size=d_ffn,
              tied=False,
              num_heads=nhead,
              fix_tm_hidden_size=False,
          )
  
      self.convolution_module = ConvolutionModule(
          d_model, kernel_size, bias, activation, dropout, causal=causal
      )
  
      self.ffn_module1 = nn.Sequential(
          nn.LayerNorm(d_model),
          PositionalwiseFeedForward(
              d_ffn=d_ffn,
              input_size=d_model,
              dropout=dropout,
              activation=activation,
          ),
          nn.Dropout(dropout),
      )
  
      self.ffn_module2 = nn.Sequential(
          nn.LayerNorm(d_model),
          PositionalwiseFeedForward(
              d_ffn=d_ffn,
              input_size=d_model,
              dropout=dropout,
              activation=activation,
          ),
          nn.Dropout(dropout),
      )
  
      self.norm1 = LayerNorm(d_model)
      self.norm2 = LayerNorm(d_model)
      self.drop = nn.Dropout(dropout)
  
imports: from dataclasses import dataclass
  import torch
  import torch.nn
  import torch.nn.functional
  from typing import Optional, List
  import speechbrain
  import warnings
  from speechbrain.nnet.attention import RelPosMHAXL, MultiheadAttention, PositionalwiseFeedForward
  from speechbrain.utils.dynamic_chunk_training import DynChunkTrainConfig
  from speechbrain.lobes.models.transformer.hypermixing import HyperMixing
  from speechbrain.nnet.normalization import LayerNorm
  from speechbrain.nnet.activations import Swish
label_desc: Use image node_id 1 to create a new ConformerEncoderLayer object from inherited base classes: nn with example: obj = ConformerEncoderLayer(d_model, d_ffn, nhead, kernel_size, kdim, vdim, activation, bias, dropout, causal, attention_type)