class_docstr: 可用于 LSTM2LSTM 的序列到序列模型的 decode 过程中，该 attention 是在 decode 过程中根据上一个 step 的 hidden 计算对 encoder 结果的 attention
  
  :param int input_size: 输入的大小
  :param int key_dim: 一般就是 encoder_output 输出的维度
  :param int value_dim: 输出的大小维度, 一般就是 decoder hidden 的大小
  :param bias:
class_docstr_tok: ["\u53ef\u7528\u4e8e","LSTM2LSTM","\u7684\u5e8f\u5217\u5230\u5e8f\u5217\u6a21\u578b\u7684","decode","\u8fc7\u7a0b\u4e2d\uff0c\u8be5","attention","\u662f\u5728","decode","\u8fc7\u7a0b\u4e2d\u6839\u636e\u4e0a\u4e00\u4e2a","step","\u7684","hidden","\u8ba1\u7b97\u5bf9","encoder","\u7ed3\u679c\u7684","attention",":","param","int","input_size",":","\u8f93\u5165\u7684\u5927\u5c0f",":","param","int","key_dim",":","\u4e00\u822c\u5c31\u662f","encoder_output","\u8f93\u51fa\u7684\u7ef4\u5ea6",":","param","int","value_dim",":","\u8f93\u51fa\u7684\u5927\u5c0f\u7ef4\u5ea6",",","\u4e00\u822c\u5c31\u662f","decoder","hidden","\u7684\u5927\u5c0f",":","param","bias",":"]
docstr: :param input: ``[batch_size, input_size]``
  :param encode_outputs: ``[batch_size, max_len, key_dim]``
  :param encode_mask: ``[batch_size, max_len]``, 为0的地方为padding
  :return: hidden: ``[batch_size, value_dim]``, scores: ``[batch_size, max_len]``, normalized 过的
docstr_tok: [":","param","input",":","``","[","batch_size",",","input_size","]","``",":","param","encode_outputs",":","``","[","batch_size",",","max_len",",","key_dim","]","``",":","param","encode_mask",":","``","[","batch_size",",","max_len","]","``",",","\u4e3a0\u7684\u5730\u65b9\u4e3apadding",":","return",":","hidden",":","``","[","batch_size",",","value_dim","]","``",",","scores",":","``","[","batch_size",",","max_len","]","``",",","normalized","\u8fc7\u7684"]
code: def forward(self, input, encode_outputs, encode_mask):
      """
  
      :param input: ``[batch_size, input_size]``
      :param encode_outputs: ``[batch_size, max_len, key_dim]``
      :param encode_mask: ``[batch_size, max_len]``, 为0的地方为padding
      :return: hidden: ``[batch_size, value_dim]``, scores: ``[batch_size, max_len]``, normalized 过的
      """
  
      # x: bsz x encode_hidden_size
      x = self.input_proj(input)
  
      # compute attention
      attn_scores = torch.matmul(
          encode_outputs, x.unsqueeze(-1)
      ).squeeze(
          -1
      )  # b x max_len
  
      # don't attend over padding
      if encode_mask is not None:
          attn_scores = (
              attn_scores.float()
              .masked_fill_(encode_mask.eq(0), float("-inf"))
              .type_as(attn_scores)
          )  # FP16 support: cast to float and back
  
      attn_scores = F.softmax(attn_scores, dim=-1)  # srclen x bsz
  
      # sum weighted sources
      x = torch.matmul(
          attn_scores.unsqueeze(1), encode_outputs
      ).squeeze(
          1
      )  # b x encode_hidden_size
  
      x = torch.tanh(self.output_proj(torch.cat((x, input), dim=1)))
      return x, attn_scores
  
imports: import math
  import torch
  import torch.nn.functional
  from torch import nn
  from .decoder.seq2seq_state import TransformerState
label_desc: Use image node_id 2 for calling the AttentionLayer obj's underlying member method code with example usage: obj.forward(input, encode_outputs, encode_mask) and returns: x,  attn_scores