class_docstr: MultiHead Attention layer.
  
  Defines the MultiHead Attention operation as described in
  [Attention Is All You Need](https://arxiv.org/abs/1706.03762) which takes
  in the tensors `query`, `key`, and `value`, and returns the dot-product attention
  between them:
  
  >>> mha = MultiHeadAttention(head_size=128, num_heads=12)
  >>> query = np.random.rand(3, 5, 4) # (batch_size, query_elements, query_depth)
  >>> key = np.random.rand(3, 6, 5) # (batch_size, key_elements, key_depth)
  >>> value = np.random.rand(3, 6, 6) # (batch_size, key_elements, value_depth)
  >>> attention = mha([query, key, value]) # (batch_size, query_elements, value_depth)
  >>> attention.shape
  TensorShape([3, 5, 6])
  
  If `value` is not given then internally `value = key` will be used:
  
  >>> mha = MultiHeadAttention(head_size=128, num_heads=12)
  >>> query = np.random.rand(3, 5, 5) # (batch_size, query_elements, query_depth)
  >>> key = np.random.rand(3, 6, 10) # (batch_size, key_elements, key_depth)
  >>> attention = mha([query, key]) # (batch_size, query_elements, key_depth)
  >>> attention.shape
  TensorShape([3, 5, 10])
  
  Args:
      head_size: int, dimensionality of the `query`, `key` and `value` tensors
          after the linear transformation.
      num_heads: int, number of attention heads.
      output_size: int, dimensionality of the output space, if `None` then the
          input dimension of `value` or `key` will be used,
          default `None`.
      dropout: float, `rate` parameter for the dropout layer that is
          applied to attention after softmax,
      default `0`.
      use_projection_bias: bool, whether to use a bias term after the linear
          output projection.
      return_attn_coef: bool, if `True`, return the attention coefficients as
          an additional output argument.
      kernel_initializer: initializer, initializer for the kernel weights.
      kernel_regularizer: regularizer, regularizer for the kernel weights.
      kernel_constraint: constraint, constraint for the kernel weights.
      bias_initializer: initializer, initializer for the bias weights.
      bias_regularizer: regularizer, regularizer for the bias weights.
      bias_constraint: constraint, constraint for the bias weights.
  
  Call Args:
      inputs:  List of `[query, key, value]` where
          * `query`: Tensor of shape `(..., query_elements, query_depth)`
          * `key`: `Tensor of shape '(..., key_elements, key_depth)`
          * `value`: Tensor of shape `(..., key_elements, value_depth)`, optional, if not given `key` will be used.
      mask: a binary Tensor of shape `[batch_size?, num_heads?, query_elements, key_elements]`
      which specifies which query elements can attendo to which key elements,
      `1` indicates attention and `0` indicates no attention.
  
  Output shape:
      * `(..., query_elements, output_size)` if `output_size` is given, else
      * `(..., query_elements, value_depth)` if `value` is given, else
      * `(..., query_elements, key_depth)`
code: def get_config(self):
      config = super().get_config()
  
      config.update(
          head_size=self.head_size,
          num_heads=self.num_heads,
          output_size=self.output_size,
          dropout=self._droput_rate,
          use_projection_bias=self.use_projection_bias,
          return_attn_coef=self.return_attn_coef,
          kernel_initializer=tf.keras.initializers.serialize(
              self.kernel_initializer
          ),
          kernel_regularizer=tf.keras.regularizers.serialize(
              self.kernel_regularizer
          ),
          kernel_constraint=tf.keras.constraints.serialize(
              self.kernel_constraint
          ),
          bias_initializer=tf.keras.initializers.serialize(
              self.bias_initializer
          ),
          bias_regularizer=tf.keras.regularizers.serialize(
              self.bias_regularizer
          ),
          bias_constraint=tf.keras.constraints.serialize(
              self.bias_constraint
          ),
      )
  
      return config
  
file_path: addons/tensorflow_addons/layers/multihead_attention.py
imports: import typing
  import warnings
  import tensorflow
label_desc: Use image node_id 5 for calling the MultiHeadAttention obj's underlying member method code with example usage: obj.get_config() and returns: config