class_docstr: Building block of Transformer encoder.
  
  Args:
      hidden_size: size of the embeddings in the model, also known as d_model
      inner_size: number of neurons in the intermediate part of feed-forward
          net, usually is (4-8 x hidden_size) in the papers
      num_attention_heads: number of heads in multi-head attention
      attn_score_dropout: probability of dropout applied to attention scores
      attn_layer_dropout: probability of dropout applied to the output of the
          attention layers, but before layer normalization
      ffn_dropout: probability of dropout applied to FFN output
      hidden_act: activation function used between two linear layers in FFN
docstr: Pre-LayerNorm block
  Order of operations: LN -> Self-Attn -> Residual -> LN -> Cross-Attn -> Residual -> LN -> FFN
docstr_tok: ["Pre-LayerNorm","block","Order","of","operations",":","LN","-",">","Self-Attn","-",">","Residual","-",">","LN","-",">","Cross-Attn","-",">","Residual","-",">","LN","-",">","FFN"]
code: def forward_preln(self, encoder_query, encoder_mask, encoder_keys):
      """
      Pre-LayerNorm block
      Order of operations: LN -> Self-Attn -> Residual -> LN -> Cross-Attn -> Residual -> LN -> FFN
      """
      residual = encoder_query
      encoder_query = self.layer_norm_1(encoder_query)
      encoder_keys = self.layer_norm_1(encoder_keys)
      self_attn_output = self.first_sub_layer(
          encoder_query, encoder_keys, encoder_keys, encoder_mask
      )
      self_attn_output += residual
  
      residual = self_attn_output
      self_attn_output = self.layer_norm_2(self_attn_output)
      output_states = self.second_sub_layer(self_attn_output)
      output_states += residual
  
      return output_states
  
file_path: NeMo/nemo/collections/nlp/modules/common/transformer/transformer_encoders.py
imports: import copy
  import torch
  import torch.nn
  from nemo.collections.common.parts import form_attention_mask
  from nemo.collections.nlp.modules.common.transformer.transformer_modules import MultiHeadAttention, PositionWiseFF
label_desc: Use image node_id 2 for calling the TransformerEncoderBlock obj's underlying member method code with example usage: obj.forward_preln(encoder_query, encoder_mask, encoder_keys) and returns: output_states