class_docstr: This module is a single transformer layer, mapping to `BertLayer` in the architecture in BERT.
  Details in the paper:
  [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, Devlin et al, 2019]
  (https://api.semanticscholar.org/CorpusID:52967399)
  
  # Parameters
  
  hidden_size : `int`
  intermediate_size : `int`
  num_attention_heads : `int`
  attention_dropout : `float` (default = `0.0`)
      Dropout probability for the `SelfAttention` layer.
  hidden_dropout : `float` (default = `0.0`)
      Dropout probability for the `OutputLayer`.
  activation : `Union[str, torch.nn.Module]`
  add_cross_attention : `bool` (default = `False`)
      If True, an extra `AttentionLayer` is added for cross-attention.
      This is helpful when using the layer in a decoder.
code: def __init__(
      self,
      hidden_size: int,
      intermediate_size: int,
      num_attention_heads: int,
      attention_dropout: float = 0.0,
      hidden_dropout: float = 0.0,
      activation: Union[str, torch.nn.Module] = "relu",
      add_cross_attention: bool = False,
  ):
      super().__init__()
  
      self.attention = AttentionLayer(
          hidden_size=hidden_size,
          num_attention_heads=num_attention_heads,
          attention_dropout=attention_dropout,
          hidden_dropout=hidden_dropout,
      )
  
      if add_cross_attention:
          self.cross_attention = AttentionLayer(
              hidden_size=hidden_size,
              num_attention_heads=num_attention_heads,
              attention_dropout=attention_dropout,
              hidden_dropout=hidden_dropout,
              is_cross_attention=True,
              is_decoder=True,
          )
  
      self.intermediate = ActivationLayer(
          hidden_size=hidden_size,
          intermediate_size=intermediate_size,
          activation=activation,
      )
      self.output = OutputLayer(
          input_size=intermediate_size,
          hidden_size=hidden_size,
          dropout=hidden_dropout,
      )
  
imports: from typing import Union, Optional, TYPE_CHECKING
  from dataclasses import dataclass
  import torch
  from allennlp.common import FromParams
  from allennlp.modules.transformer.transformer_module import TransformerModule
  from allennlp.modules.transformer.activation_layer import ActivationLayer
  from allennlp.modules.transformer.attention_module import SelfAttention, AttentionOutput
  from allennlp.modules.transformer.output_layer import OutputLayer
  from allennlp.modules.transformer.util import FloatT
label_desc: Use image node_id 1 to create a new TransformerLayer object from inherited base classes: TransformerModule, FromParams with example: obj = TransformerLayer(hidden_size, intermediate_size, num_attention_heads, attention_dropout, hidden_dropout, activation, add_cross_attention)