class_docstr: <a id="TransformerLayer"></a>
  
  ## Transformer Layer
  
  This can act as an encoder layer or a decoder layer.
  
  🗒 Some implementations, including the paper seem to have differences
  in where the layer-normalization is done.
  Here we do a layer normalization before attention and feed-forward networks,
  and add the original residual vectors.
  Alternative is to do a layer normalization after adding the residuals.
  But we found this to be less stable when training.
  We found a detailed discussion about this in the paper
   [On Layer Normalization in the Transformer Architecture](https://arxiv.org/abs/2002.04745).
code: def forward(
      self,
      *,
      x: torch.Tensor,
      mask: torch.Tensor,
      src: torch.Tensor = None,
      src_mask: torch.Tensor = None,
  ):
      # Normalize the vectors before doing self attention
      z = self.norm_self_attn(x)
      # Run through self attention, i.e. keys and values are from self
      self_attn = self.self_attn(query=z, key=z, value=z, mask=mask)
      # Add the self attention results
      x = x + self.dropout(self_attn)
  
      # If a source is provided, get results from attention to source.
      # This is when you have a decoder layer that pays attention to
      # encoder outputs
      if src is not None:
          # Normalize vectors
          z = self.norm_src_attn(x)
          # Attention to source. i.e. keys and values are from source
          attn_src = self.src_attn(
              query=z, key=src, value=src, mask=src_mask
          )
          # Add the source attention results
          x = x + self.dropout(attn_src)
  
      # Normalize for feed-forward
      z = self.norm_ff(x)
      # Save the input to the feed forward layer if specified
      if self.is_save_ff_input:
          self.ff_input = z.clone()
      # Pass through the feed-forward network
      ff = self.feed_forward(z)
      # Add the feed-forward results back
      x = x + self.dropout(ff)
  
      return x
  
imports: import math
  import torch
  import torch.nn
  from labml_nn.utils import clone_module_list
  from .feed_forward import FeedForward
  from .mha import MultiHeadAttention
  from .positional_encoding import get_positional_encoding
label_desc: Use image node_id 2 for calling the TransformerLayer obj's underlying member method code with example usage: obj.forward() and returns: x