class_docstr: This class implements the key-value scaled dot product attention mechanism
  detailed in the paper `Attention is all you Need
  <https://www.semanticscholar.org/paper/Attention-Is-All-You-Need-Vaswani-Shazeer/0737da0767d77606169cbf4187b83e1ab62f6077>`_ .
  
  The attention mechanism is a weighted sum of a projection V of the inputs, with respect
  to the scaled, normalised dot product of Q and K, which are also both linear projections
  of the input. This procedure is repeated for each attention head, using different parameters.
  
  Parameters
  ----------
  num_heads : ``int``, required.
      The number of attention heads to use.
  input_dim : ``int``, required.
      The size of the last dimension of the input tensor.
  attention_dim ``int``, required.
      The total dimension of the query and key projections which comprise the
      dot product attention function. Must be divisible by ``num_heads``.
  values_dim : ``int``, required.
      The total dimension which the input is projected to for representing the values,
      which are combined using the attention. Must be divisible by ``num_heads``.
  output_projection_dim : ``int``, optional (default = None)
      The dimensionality of the final output projection. If this is not passed
      explicitly, the projection has size `input_size`.
  attention_dropout_prob : ``float``, optional (default = 0.1).
      The dropout probability applied to the normalised attention
      distributions.
class_docstr_tok: ["This","class","implements","the","key-value","scaled","dot","product","attention","mechanism","detailed","in","the","paper","`","Attention","is","all","you","Need","<","https",":","\/\/www.semanticscholar.org\/paper\/Attention-Is-All-You-Need-Vaswani-Shazeer\/0737da0767d77606169cbf4187b83e1ab62f6077",">","`","_",".","The","attention","mechanism","is","a","weighted","sum","of","a","projection","V","of","the","inputs",",","with","respect","to","the","scaled",",","normalised","dot","product","of","Q","and","K",",","which","are","also","both","linear","projections","of","the","input",".","This","procedure","is","repeated","for","each","attention","head",",","using","different","parameters",".","Parameters","--","--","--","--","--","num_heads",":","``","int","``",",","required",".","The","number","of","attention","heads","to","use",".","input_dim",":","``","int","``",",","required",".","The","size","of","the","last","dimension","of","the","input","tensor",".","attention_dim","``","int","``",",","required",".","The","total","dimension","of","the","query","and","key","projections","which","comprise","the","dot","product","attention","function",".","Must","be","divisible","by","``","num_heads","``",".","values_dim",":","``","int","``",",","required",".","The","total","dimension","which","the","input","is","projected","to","for","representing","the","values",",","which","are","combined","using","the","attention",".","Must","be","divisible","by","``","num_heads","``",".","output_projection_dim",":","``","int","``",",","optional","(","default","=","None",")","The","dimensionality","of","the","final","output","projection",".","If","this","is","not","passed","explicitly",",","the","projection","has","size","`","input_size","`",".","attention_dropout_prob",":","``","float","``",",","optional","(","default","=","0.1",")",".","The","dropout","probability","applied","to","the","normalised","attention","distributions","."]
code: def get_input_dim(self):
      return self._input_dim
  
file_path: magnitude/pymagnitude/third_party/allennlp/modules/seq2seq_encoders/multi_head_self_attention.py
imports: from __future__ import division
  from __future__ import absolute_import
  import torch
  from torch.nn import Dropout, Linear
  from allennlp.nn.util import last_dim_softmax, weighted_sum
  from allennlp.modules.seq2seq_encoders.seq2seq_encoder import Seq2SeqEncoder
label_desc: Use image node_id 2 for calling the MultiHeadSelfAttention obj's underlying member method code with example usage: obj.get_input_dim() and returns: self