class_docstr: Computes attention for two modalities, based on
  [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations
  for Vision-and-Language Tasks (Lu et al, 2019)]
  (https://api.semanticscholar.org/CorpusID:199453025).
  
  From the paper:
  
  "The keys and values from each modality are passed as input to the
  other modality’s multi-headed attention block. Consequentially, the
  attention block produces attention-pooled features for each modality
  conditioned on the other."
  
  For example, considering the case when the first modality is image,
  and the second modality is language, the module performs
  "image-conditioned language attention in the visual stream and
  language-conditioned image attention in the linguistic stream."
  
  # Parameters
  
  hidden_size1 : `int`
      The input hidden dim for the first modality.
  hidden_size2 : `int`
      The input hidden dim for the second modality.
  combined_hidden_size : `int`
      The output hidden dim for both modalities; it should be a multiple
      of `num_attention_heads`.
  num_attention_heads : `int`
      The number of attention heads.
  dropout1 : `float` (default = `0.0`)
      The dropout probability for the first modality stream.
  dropout2 : `float` (default = `0.0`)
      The dropout probability for the second modality stream.
  scoring_func1 : `str` (default = `scaled_dot_product`)
      The name of the attention-calculating function to be used for the first modality.
  scoring_func2 : `str` (default = `scaled_dot_product`)
      The name of the attention-calculating function to be used for the second modality.
      Eg. `dot_product`, `linear`, etc. For a complete list, please check
      :mod:`allennlp.modules.matrix_attention`.
code: def __init__(
      self,
      hidden_size1: int,
      hidden_size2: int,
      combined_hidden_size: int,
      num_attention_heads: int,
      dropout1: float = 0.0,
      dropout2: float = 0.0,
      scoring_func1: str = "scaled_dot_product",
      scoring_func2: str = "scaled_dot_product",
  ):
      super().__init__()
      if combined_hidden_size % num_attention_heads != 0:
          raise ValueError(
              "The hidden size (%d) is not a multiple of the number of attention "
              "heads (%d)" % (combined_hidden_size, num_attention_heads)
          )
  
      self.num_attention_heads = num_attention_heads
      self.attention_head_size = int(
          combined_hidden_size / num_attention_heads
      )
  
      # This is basically the `combined_hidden_size`, since we already ensure
      # that `combined_hidden_size` is divisible by `num_attention_heads`.
      self.all_head_size = (
          self.num_attention_heads * self.attention_head_size
      )
  
      # First modality:
  
      self.query1 = torch.nn.Linear(hidden_size1, self.all_head_size)
      self.key1 = torch.nn.Linear(hidden_size1, self.all_head_size)
      self.value1 = torch.nn.Linear(hidden_size1, self.all_head_size)
  
      self.scoring_func1 = scoring_func1
      self.attn1 = MatrixAttention.by_name(self.scoring_func1)()
      self.dropout1 = torch.nn.Dropout(dropout1)
  
      # Second modality:
  
      self.query2 = torch.nn.Linear(hidden_size2, self.all_head_size)
      self.key2 = torch.nn.Linear(hidden_size2, self.all_head_size)
      self.value2 = torch.nn.Linear(hidden_size2, self.all_head_size)
  
      self.scoring_func2 = scoring_func2
      self.attn2 = MatrixAttention.by_name(self.scoring_func2)()
      self.dropout2 = torch.nn.Dropout(dropout2)
  
file_path: allennlp/allennlp/modules/transformer/bimodal_attention.py
imports: import torch
  from allennlp.common import FromParams
  from allennlp.modules.matrix_attention.matrix_attention import MatrixAttention
  from allennlp.modules.transformer.transformer_module import TransformerModule
  from allennlp.modules.transformer.util import apply_mask
label_desc: Use image node_id 1 to create a new BiModalAttention object from inherited base classes: TransformerModule, FromParams with example: obj = BiModalAttention(hidden_size1, hidden_size2, combined_hidden_size, num_attention_heads, dropout1, dropout2, scoring_func1, scoring_func2)