class_docstr: The original attention from Badhanau et al. (2014)
  
  https://arxiv.org/abs/1409.0473, based on a Multi-Layer Perceptron.
  The attention score between position i in the encoder and position j in the
  decoder is: alpha_ij = V_a * tanh(W_ae * enc_i + W_ad * dec_j + b_a)
class_docstr_tok: ["The","original","attention","from","Badhanau","et","al",".","(","2014",")","https",":","\/\/arxiv.org\/abs\/1409.0473",",","based","on","a","Multi-Layer","Perceptron",".","The","attention","score","between","position","i","in","the","encoder","and","position","j","in","the","decoder","is",":","alpha_ij","=","V_a","*","tanh","(","W_ae","*","enc_i","+","W_ad","*","dec_j","+","b_a",")"]
docstr: The expected input dimensions are:
  decoder_state: bsz x decoder_hidden_state_dim
  source_hids: src_len x bsz x context_dim
  encoder_padding_mask: src_len x bsz
docstr_tok: ["The","expected","input","dimensions","are",":","decoder_state",":","bsz","x","decoder_hidden_state_dim","source_hids",":","src_len","x","bsz","x","context_dim","encoder_padding_mask",":","src_len","x","bsz"]
code: def forward(self, decoder_state, source_hids, encoder_padding_mask):
      """The expected input dimensions are:
      decoder_state: bsz x decoder_hidden_state_dim
      source_hids: src_len x bsz x context_dim
      encoder_padding_mask: src_len x bsz
      """
      src_len, bsz, _ = source_hids.size()
      # (src_len*bsz) x context_dim (to feed through linear)
      flat_source_hids = source_hids.view(-1, self.context_dim)
      # (src_len*bsz) x attention_dim
      encoder_component = self.encoder_proj(flat_source_hids)
      # src_len x bsz x attention_dim
      encoder_component = encoder_component.view(
          src_len, bsz, self.attention_dim
      )
      # 1 x bsz x attention_dim
      decoder_component = self.decoder_proj(decoder_state).unsqueeze(0)
      # Sum with broadcasting and apply the non linearity
      # src_len x bsz x attention_dim
      hidden_att = torch.tanh(
          (decoder_component + encoder_component).view(
              -1, self.attention_dim
          )
      )
      # Project onto the reals to get attentions scores (src_len x bsz)
      attn_scores = self.to_scores(hidden_att).view(src_len, bsz)
  
      # Mask + softmax (src_len x bsz)
      if encoder_padding_mask is not None:
          attn_scores = (
              attn_scores.float()
              .masked_fill_(encoder_padding_mask, float("-inf"))
              .type_as(attn_scores)
          )  # FP16 support: cast to float and back
      # srclen x bsz
      normalized_masked_attn_scores = F.softmax(attn_scores, dim=0)
  
      # Sum weighted sources (bsz x context_dim)
      attn_weighted_context = (
          source_hids * normalized_masked_attn_scores.unsqueeze(2)
      ).sum(dim=0)
  
      return attn_weighted_context, normalized_masked_attn_scores
  
imports: from ast import literal_eval
  from typing import List, Tuple
  import torch
  import torch.nn
  import torch.nn.functional
  from fairseq import checkpoint_utils, utils
  from fairseq.data.data_utils import lengths_to_padding_mask
  from fairseq.models import FairseqEncoder, FairseqEncoderDecoderModel, FairseqIncrementalDecoder, register_model, register_model_architecture
label_desc: Use image node_id 2 for calling the MLPAttention obj's underlying member method code with example usage: obj.forward(decoder_state, source_hids, encoder_padding_mask) and returns: attn_weighted_context,  normalized_masked_attn_scores