class_docstr: 这是一个基于论文 `A structured self-attentive sentence embedding <https://arxiv.org/pdf/1703.03130.pdf>`_
  的 **Self Attention Module** 。
  
  :param  input_size: 输入 tensor 的 hidden 维度
  :param attention_unit: 输出 tensor 的 hidden 维度
  :param attention_hops:
  :param drop: dropout 概率
class_docstr_tok: ["\u8fd9\u662f\u4e00\u4e2a\u57fa\u4e8e\u8bba\u6587","`","A","structured","self-attentive","sentence","embedding","<","https",":","\/\/arxiv.org\/pdf\/1703.03130.pdf",">","`","_","\u7684","*","*","Self","Attention","Module","*","*","\u3002",":","param","input_size",":","\u8f93\u5165","tensor","\u7684","hidden","\u7ef4\u5ea6",":","param","attention_unit",":","\u8f93\u51fa","tensor","\u7684","hidden","\u7ef4\u5ea6",":","param","attention_hops",":",":","param","drop",":","dropout","\u6982\u7387"]
docstr: :param input: 要做 **attention** 的矩阵，形状为 ``[batch_size, seq_len, hidden_size]``
  :param input_origin: 原始 token 的 index 组成的矩阵，含有 pad 部分内容，形状为 ``[batch_size, seq_len]`` 
  :return: 一个元组，分别是：
  
          - 经过 **attention** 操作后输入矩阵的结果，形状为 ``[batch_size, multi-head, hidden_size]``
          - **attention** 惩罚项，是一个标量
docstr_tok: [":","param","input",":","\u8981\u505a","*","*","attention","*","*","\u7684\u77e9\u9635\uff0c\u5f62\u72b6\u4e3a","``","[","batch_size",",","seq_len",",","hidden_size","]","``",":","param","input_origin",":","\u539f\u59cb","token","\u7684","index","\u7ec4\u6210\u7684\u77e9\u9635\uff0c\u542b\u6709","pad","\u90e8\u5206\u5185\u5bb9\uff0c\u5f62\u72b6\u4e3a","``","[","batch_size",",","seq_len","]","``",":","return",":","\u4e00\u4e2a\u5143\u7ec4\uff0c\u5206\u522b\u662f\uff1a","-","\u7ecf\u8fc7","*","*","attention","*","*","\u64cd\u4f5c\u540e\u8f93\u5165\u77e9\u9635\u7684\u7ed3\u679c\uff0c\u5f62\u72b6\u4e3a","``","[","batch_size",",","multi-head",",","hidden_size","]","``","-","*","*","attention","*","*","\u60e9\u7f5a\u9879\uff0c\u662f\u4e00\u4e2a\u6807\u91cf"]
code: def forward(self, input, input_origin):
      r"""
      :param input: 要做 **attention** 的矩阵，形状为 ``[batch_size, seq_len, hidden_size]``
      :param input_origin: 原始 token 的 index 组成的矩阵，含有 pad 部分内容，形状为 ``[batch_size, seq_len]``
      :return: 一个元组，分别是：
  
              - 经过 **attention** 操作后输入矩阵的结果，形状为 ``[batch_size, multi-head, hidden_size]``
              - **attention** 惩罚项，是一个标量
      """
      input = input.contiguous()
      size = input.size()  # [bsz, len, nhid]
  
      input_origin = input_origin.expand(
          self.attention_hops, -1, -1
      )  # [hops,baz, len]
      input_origin = input_origin.transpose(
          0, 1
      ).contiguous()  # [baz, hops,len]
  
      y1 = self.tanh(
          self.ws1(self.drop(input))
      )  # [baz,len,dim] -->[bsz,len, attention-unit]
      attention = self.ws2(y1).transpose(1, 2).contiguous()
      # [bsz,len, attention-unit]--> [bsz, len, hop]--> [baz,hop,len]
  
      attention = attention + (
          -999999 * (input_origin == 0).float()
      )  # remove the weight on padding token.
      attention = F.softmax(attention, 2)  # [baz ,hop, len]
      return torch.bmm(attention, input), self._penalization(attention)
  
imports: import math
  import torch
  import torch.nn.functional
  from torch import nn
  from .decoder.seq2seq_state import TransformerState
label_desc: Use image node_id 3 for calling the SelfAttention obj's underlying member method code with example usage: obj.forward(input, input_origin) and returns: torch,  self