class_docstr: - Q (query), K (key) and V (value) are split into multiple heads (num_heads)
  - each tuple (q, k, v) are fed to scaled_dot_product_attention
  - all attention outputs are concatenated
docstr: Initialize parameters.
  
  Args:
      attention_dim (int): Dimension of the attention embeddings.
      num_heads (int): Number of heads in the multi-head self-attention module.
      dropout_rate (float): Dropout probability.
docstr_tok: ["Initialize","parameters",".","Args",":","attention_dim","(","int",")",":","Dimension","of","the","attention","embeddings",".","num_heads","(","int",")",":","Number","of","heads","in","the","multi-head","self-attention","module",".","dropout_rate","(","float",")",":","Dropout","probability","."]
code: def __init__(self, attention_dim, num_heads, dropout_rate):
      """Initialize parameters.
  
      Args:
          attention_dim (int): Dimension of the attention embeddings.
          num_heads (int): Number of heads in the multi-head self-attention module.
          dropout_rate (float): Dropout probability.
      """
      super(MultiHeadAttention, self).__init__()
      self.num_heads = num_heads
      self.attention_dim = attention_dim
      assert attention_dim % self.num_heads == 0
      self.dropout_rate = dropout_rate
  
      self.depth = attention_dim // self.num_heads
  
      self.Q = tf.keras.layers.Dense(
          self.attention_dim, activation=None
      )
      self.K = tf.keras.layers.Dense(
          self.attention_dim, activation=None
      )
      self.V = tf.keras.layers.Dense(
          self.attention_dim, activation=None
      )
      self.dropout = tf.keras.layers.Dropout(self.dropout_rate)
  
imports: import random
  import numpy
  from tqdm import tqdm
  import tensorflow
  from recommenders.utils.timer import Timer
label_desc: Use image node_id 1 to create a new MultiHeadAttention object from inherited base classes: tf.keras.layers with example: obj = MultiHeadAttention(attention_dim, num_heads, dropout_rate)