class_docstr: The multihead self-attention layer that transforms a set of matrices
  according to a formula:
  
  :math:`Q = W_Q * Q`,
  :math:`K = W_K * K`,
  :math:`V = W_V * V`
  where :math:`W_*` are trainable matrices of size (Channels_* x GetHiddenSize())
  
  :math:`Attention(Q, K, V) = softmax( Q * K_t / \sqrt{d_K} ) * V`
  where :math:`d_k` - dimension of k
  
  :math:`MultiHeadAttention = dropout\_if\_needed(concat( head_1, ..., head_N )) * W_O`
  where :math:`head_i = Attention( W_{Q,i} * X, W_{K,i} * X, W_{V,i} * X )`, 
  :math:`W_*` - trainable parameters and :math:`W_O` is an additional trainable matrix of size (GetHiddenSize() x GetOutputSize())
  
  See the papers: https://arxiv.org/pdf/1706.03762.pdf
  https://arxiv.org/pdf/1807.03819.pdf
  
  :param input_layer: The input layers to be connected. 
      The integer in each tuple specifies the number of the output.
      If not set, the first output will be used.
  :type input_layer: list of object, tuple(object, int)
  :param head_count: The number of heads in attention.
  :type head_count: int, > 0, default=1
  :param hidden_size: The size of trainable matrices W_*. Should be a multiple of head_count.
  :type hidden_size: int, > 0     
  :param output_size: The size of output.
  :type output_size: int, > 0
  :param dropout_rate: Rate of droupout applied to the softmax. 
      A negative value means no dropout.
  :type dropout_rate: float, default=-1
  :param name: The layer name.
  :type name: str, default=None
  
  .. rubric:: Layer inputs:
  
  (1) matrix Q of shape (1 x **BatchWidth** x **ListSize_Q** x 1 x 1 x 1 x **Channels_Q**)
  (2) matrix K of shape (1 x **BatchWidth** x **ListSize_V** x 1 x 1 x 1 x **Channels_Q**)
  (3) matrix V (1 x **BatchWidth** x **ListSize_V** x 1 x 1 x 1 x **Channels_V**)
  (4) (optional): mask, can be 0.
  
  .. rubric:: Layer outputs:
  
  (1) result matrix of shape (1, **BatchWidth**, **ListSize_Q**, 1, 1, 1, output_size)
docstr: Gets the rate of dropout applied to the softmax.
          
docstr_tok: ["Gets","the","rate","of","dropout","applied","to","the","softmax","."]
code: def dropout_rate(self):
      """Gets the rate of dropout applied to the softmax."""
      return self._internal.get_dropout_rate()
  
file_path: neoml/NeoML/Python/neoml/Dnn/MultiheadAttention.py
imports: import neoml.PythonWrapper
  from .Dnn import Layer
  from neoml.Utils import check_input_layers
label_desc: Use image node_id 8 for calling the MultiheadAttention obj's underlying member method code with example usage: obj.dropout_rate() and returns: self