docstr: nn.LayerNorm is applied either before or after the self-attention/ffn modules similar to the original
  Transformer implementation.
docstr_tok: ["nn.LayerNorm","is","applied","either","before","or","after","the","self-attention\/ffn","modules","similar","to","the","original","Transformer","implementation","."]
code: def forward(
      self,
      input_nodes: torch.Tensor,
      self_attn_bias: Optional[torch.Tensor] = None,
      self_attn_mask: Optional[torch.Tensor] = None,
      self_attn_padding_mask: Optional[torch.Tensor] = None,
  ) -> Tuple[torch.Tensor, Optional[torch.Tensor]]:
      """
      nn.LayerNorm is applied either before or after the self-attention/ffn modules similar to the original
      Transformer implementation.
      """
      residual = input_nodes
      if self.pre_layernorm:
          input_nodes = self.self_attn_layer_norm(input_nodes)
  
      input_nodes, attn = self.self_attn(
          query=input_nodes,
          key=input_nodes,
          value=input_nodes,
          attn_bias=self_attn_bias,
          key_padding_mask=self_attn_padding_mask,
          need_weights=False,
          attn_mask=self_attn_mask,
      )
      input_nodes = self.dropout_module(input_nodes)
      input_nodes = residual + input_nodes
      if not self.pre_layernorm:
          input_nodes = self.self_attn_layer_norm(input_nodes)
  
      residual = input_nodes
      if self.pre_layernorm:
          input_nodes = self.final_layer_norm(input_nodes)
      input_nodes = self.activation_fn(self.fc1(input_nodes))
      input_nodes = self.activation_dropout_module(input_nodes)
      input_nodes = self.fc2(input_nodes)
      input_nodes = self.dropout_module(input_nodes)
      input_nodes = residual + input_nodes
      if not self.pre_layernorm:
          input_nodes = self.final_layer_norm(input_nodes)
  
      return input_nodes, attn
  
imports: import math
  from typing import Iterable, Iterator, List, Optional, Tuple, Union
  import torch
  import torch.nn
  from torch.nn import BCEWithLogitsLoss, CrossEntropyLoss, MSELoss
  from ...activations import ACT2FN
  from ...modeling_outputs import BaseModelOutputWithNoAttention, SequenceClassifierOutput
  from ...modeling_utils import PreTrainedModel
  from ...utils import logging
  from .configuration_graphormer import GraphormerConfig
label_desc: Use image node_id 3 for calling the GraphormerGraphEncoderLayer obj's underlying member method code with example usage: obj.forward(input_nodes, _attn_bias, _attn_mask, _attn_padding_mask) and returns: input_nodes,  attn