class_docstr: This ``Module`` implements the matching layer of BiMPM model described in `Bilateral
  Multi-Perspective Matching for Natural Language Sentences <https://arxiv.org/abs/1702.03814>`_
  by Zhiguo Wang et al., 2017.
  Also please refer to the `TensorFlow implementation <https://github.com/zhiguowang/BiMPM/>`_ and
  `PyTorch implementation <https://github.com/galsang/BIMPM-pytorch>`_.
  
  Parameters
  ----------
  hidden_dim : ``int``, optional (default = 100)
      The hidden dimension of the representations
  num_perspectives : ``int``, optional (default = 20)
      The number of perspectives for matching
  share_weights_between_directions : ``bool``, optional (default = True)
      If True, share weight between matching from sentence1 to sentence2 and from sentence2
      to sentence1, useful for non-symmetric tasks
  is_forward : ``bool``, optional (default = None)
      Whether the matching is for forward sequence or backward sequence, useful in finding last
      token in full matching. It can not be None if with_full_match is True.
  with_full_match : ``bool``, optional (default = True)
      If True, include full match
  with_maxpool_match : ``bool``, optional (default = True)
      If True, include max pool match
  with_attentive_match : ``bool``, optional (default = True)
      If True, include attentive match
  with_max_attentive_match : ``bool``, optional (default = True)
      If True, include max attentive match
class_docstr_tok: ["This","``","Module","``","implements","the","matching","layer","of","BiMPM","model","described","in","`","Bilateral","Multi-Perspective","Matching","for","Natural","Language","Sentences","<","https",":","\/\/arxiv.org\/abs\/1702.03814",">","`","_","by","Zhiguo","Wang","et","al.",",","2017",".","Also","please","refer","to","the","`","TensorFlow","implementation","<","https",":","\/\/github.com\/zhiguowang\/BiMPM\/",">","`","_","and","`","PyTorch","implementation","<","https",":","\/\/github.com\/galsang\/BIMPM-pytorch",">","`","_",".","Parameters","--","--","--","--","--","hidden_dim",":","``","int","``",",","optional","(","default","=","100",")","The","hidden","dimension","of","the","representations","num_perspectives",":","``","int","``",",","optional","(","default","=","20",")","The","number","of","perspectives","for","matching","share_weights_between_directions",":","``","bool","``",",","optional","(","default","=","True",")","If","True",",","share","weight","between","matching","from","sentence1","to","sentence2","and","from","sentence2","to","sentence1",",","useful","for","non-symmetric","tasks","is_forward",":","``","bool","``",",","optional","(","default","=","None",")","Whether","the","matching","is","for","forward","sequence","or","backward","sequence",",","useful","in","finding","last","token","in","full","matching",".","It","can","not","be","None","if","with_full_match","is","True",".","with_full_match",":","``","bool","``",",","optional","(","default","=","True",")","If","True",",","include","full","match","with_maxpool_match",":","``","bool","``",",","optional","(","default","=","True",")","If","True",",","include","max","pool","match","with_attentive_match",":","``","bool","``",",","optional","(","default","=","True",")","If","True",",","include","attentive","match","with_max_attentive_match",":","``","bool","``",",","optional","(","default","=","True",")","If","True",",","include","max","attentive","match"]
docstr: Given the forward (or backward) representations of sentence1 and sentence2, apply four bilateral
  matching functions between them in one direction.
  
  Parameters
  ----------
  context_1 : ``torch.Tensor``
      Tensor of shape (batch_size, seq_len1, hidden_dim) representing the encoding of the first sentence.
  mask_1 : ``torch.Tensor``
      Binary Tensor of shape (batch_size, seq_len1), indicating which
      positions in the first sentence are padding (0) and which are not (1).
  context_2 : ``torch.Tensor``
      Tensor of shape (batch_size, seq_len2, hidden_dim) representing the encoding of the second sentence.
  mask_2 : ``torch.Tensor``
      Binary Tensor of shape (batch_size, seq_len2), indicating which
      positions in the second sentence are padding (0) and which are not (1).
  
  Returns
  -------
  A tuple of matching vectors for the two sentences. Each of which is a list of
  matching vectors of shape (batch, seq_len, num_perspectives or 1)
docstr_tok: ["Given","the","forward","(","or","backward",")","representations","of","sentence1","and","sentence2",",","apply","four","bilateral","matching","functions","between","them","in","one","direction",".","Parameters","--","--","--","--","--","context_1",":","``","torch.Tensor","``","Tensor","of","shape","(","batch_size",",","seq_len1",",","hidden_dim",")","representing","the","encoding","of","the","first","sentence",".","mask_1",":","``","torch.Tensor","``","Binary","Tensor","of","shape","(","batch_size",",","seq_len1",")",",","indicating","which","positions","in","the","first","sentence","are","padding","(","0",")","and","which","are","not","(","1",")",".","context_2",":","``","torch.Tensor","``","Tensor","of","shape","(","batch_size",",","seq_len2",",","hidden_dim",")","representing","the","encoding","of","the","second","sentence",".","mask_2",":","``","torch.Tensor","``","Binary","Tensor","of","shape","(","batch_size",",","seq_len2",")",",","indicating","which","positions","in","the","second","sentence","are","padding","(","0",")","and","which","are","not","(","1",")",".","Returns","--","--","--","-","A","tuple","of","matching","vectors","for","the","two","sentences",".","Each","of","which","is","a","list","of","matching","vectors","of","shape","(","batch",",","seq_len",",","num_perspectives","or","1",")"]
code: def forward(self, context_1, mask_1, context_2, mask_2):
      # pylint: disable=arguments-differ
      """
      Given the forward (or backward) representations of sentence1 and sentence2, apply four bilateral
      matching functions between them in one direction.
  
      Parameters
      ----------
      context_1 : ``torch.Tensor``
          Tensor of shape (batch_size, seq_len1, hidden_dim) representing the encoding of the first sentence.
      mask_1 : ``torch.Tensor``
          Binary Tensor of shape (batch_size, seq_len1), indicating which
          positions in the first sentence are padding (0) and which are not (1).
      context_2 : ``torch.Tensor``
          Tensor of shape (batch_size, seq_len2, hidden_dim) representing the encoding of the second sentence.
      mask_2 : ``torch.Tensor``
          Binary Tensor of shape (batch_size, seq_len2), indicating which
          positions in the second sentence are padding (0) and which are not (1).
  
      Returns
      -------
      A tuple of matching vectors for the two sentences. Each of which is a list of
      matching vectors of shape (batch, seq_len, num_perspectives or 1)
      """
      assert (not mask_2.requires_grad) and (not mask_1.requires_grad)
      assert context_1.size(-1) == context_2.size(-1) == self.hidden_dim
  
      # (batch,)
      len_1 = get_lengths_from_binary_sequence_mask(mask_1)
      len_2 = get_lengths_from_binary_sequence_mask(mask_2)
  
      # (batch, seq_len*)
      mask_1, mask_2 = mask_1.float(), mask_2.float()
  
      # explicitly set masked weights to zero
      # (batch_size, seq_len*, hidden_dim)
      context_1 = context_1 * mask_1.unsqueeze(-1)
      context_2 = context_2 * mask_2.unsqueeze(-1)
  
      # array to keep the matching vectors for the two sentences
      matching_vector_1 = []
      matching_vector_2 = []
  
      # Step 0. unweighted cosine
      # First calculate the cosine similarities between each forward
      # (or backward) contextual embedding and every forward (or backward)
      # contextual embedding of the other sentence.
  
      # (batch, seq_len1, seq_len2)
      cosine_sim = F.cosine_similarity(
          context_1.unsqueeze(-2), context_2.unsqueeze(-3), dim=3
      )
  
      # (batch, seq_len*, 1)
      cosine_max_1 = masked_max(
          cosine_sim, mask_2.unsqueeze(-2), dim=2, keepdim=True
      )
      cosine_mean_1 = masked_mean(
          cosine_sim, mask_2.unsqueeze(-2), dim=2, keepdim=True
      )
      cosine_max_2 = masked_max(
          cosine_sim.permute(0, 2, 1),
          mask_1.unsqueeze(-2),
          dim=2,
          keepdim=True,
      )
      cosine_mean_2 = masked_mean(
          cosine_sim.permute(0, 2, 1),
          mask_1.unsqueeze(-2),
          dim=2,
          keepdim=True,
      )
  
      matching_vector_1.extend([cosine_max_1, cosine_mean_1])
      matching_vector_2.extend([cosine_max_2, cosine_mean_2])
  
      # Step 1. Full-Matching
      # Each time step of forward (or backward) contextual embedding of one sentence
      # is compared with the last time step of the forward (or backward)
      # contextual embedding of the other sentence
      if self.with_full_match:
          # (batch, 1, hidden_dim)
          if self.is_forward:
              # (batch, 1, hidden_dim)
              last_position_1 = (len_1 - 1).clamp(min=0)
              last_position_1 = last_position_1.view(-1, 1, 1).expand(
                  -1, 1, self.hidden_dim
              )
              last_position_2 = (len_2 - 1).clamp(min=0)
              last_position_2 = last_position_2.view(-1, 1, 1).expand(
                  -1, 1, self.hidden_dim
              )
  
              context_1_last = context_1.gather(1, last_position_1)
              context_2_last = context_2.gather(1, last_position_2)
          else:
              context_1_last = context_1[:, 0:1, :]
              context_2_last = context_2[:, 0:1, :]
  
          # (batch, seq_len*, num_perspectives)
          matching_vector_1_full = multi_perspective_match(
              context_1, context_2_last, self.full_match_weights
          )
          matching_vector_2_full = multi_perspective_match(
              context_2,
              context_1_last,
              self.full_match_weights_reversed,
          )
  
          matching_vector_1.extend(matching_vector_1_full)
          matching_vector_2.extend(matching_vector_2_full)
  
      # Step 2. Maxpooling-Matching
      # Each time step of forward (or backward) contextual embedding of one sentence
      # is compared with every time step of the forward (or backward)
      # contextual embedding of the other sentence, and only the max value of each
      # dimension is retained.
      if self.with_maxpool_match:
          # (batch, seq_len1, seq_len2, num_perspectives)
          matching_vector_max = multi_perspective_match_pairwise(
              context_1, context_2, self.maxpool_match_weights
          )
  
          # (batch, seq_len*, num_perspectives)
          matching_vector_1_max = masked_max(
              matching_vector_max,
              mask_2.unsqueeze(-2).unsqueeze(-1),
              dim=2,
          )
          matching_vector_1_mean = masked_mean(
              matching_vector_max,
              mask_2.unsqueeze(-2).unsqueeze(-1),
              dim=2,
          )
          matching_vector_2_max = masked_max(
              matching_vector_max.permute(0, 2, 1, 3),
              mask_1.unsqueeze(-2).unsqueeze(-1),
              dim=2,
          )
          matching_vector_2_mean = masked_mean(
              matching_vector_max.permute(0, 2, 1, 3),
              mask_1.unsqueeze(-2).unsqueeze(-1),
              dim=2,
          )
  
          matching_vector_1.extend(
              [matching_vector_1_max, matching_vector_1_mean]
          )
          matching_vector_2.extend(
              [matching_vector_2_max, matching_vector_2_mean]
          )
  
      # Step 3. Attentive-Matching
      # Each forward (or backward) similarity is taken as the weight
      # of the forward (or backward) contextual embedding, and calculate an
      # attentive vector for the sentence by weighted summing all its
      # contextual embeddings.
      # Finally match each forward (or backward) contextual embedding
      # with its corresponding attentive vector.
  
      # (batch, seq_len1, seq_len2, hidden_dim)
      att_2 = context_2.unsqueeze(-3) * cosine_sim.unsqueeze(-1)
  
      # (batch, seq_len1, seq_len2, hidden_dim)
      att_1 = context_1.unsqueeze(-2) * cosine_sim.unsqueeze(-1)
  
      if self.with_attentive_match:
          # (batch, seq_len*, hidden_dim)
          att_mean_2 = masked_softmax(
              att_2.sum(dim=2), mask_1.unsqueeze(-1)
          )
          att_mean_1 = masked_softmax(
              att_1.sum(dim=1), mask_2.unsqueeze(-1)
          )
  
          # (batch, seq_len*, num_perspectives)
          matching_vector_1_att_mean = multi_perspective_match(
              context_1, att_mean_2, self.attentive_match_weights
          )
          matching_vector_2_att_mean = multi_perspective_match(
              context_2,
              att_mean_1,
              self.attentive_match_weights_reversed,
          )
          matching_vector_1.extend(matching_vector_1_att_mean)
          matching_vector_2.extend(matching_vector_2_att_mean)
  
      # Step 4. Max-Attentive-Matching
      # Pick the contextual embeddings with the highest cosine similarity as the attentive
      # vector, and match each forward (or backward) contextual embedding with its
      # corresponding attentive vector.
      if self.with_max_attentive_match:
          # (batch, seq_len*, hidden_dim)
          att_max_2 = masked_max(
              att_2, mask_2.unsqueeze(-2).unsqueeze(-1), dim=2
          )
          att_max_1 = masked_max(
              att_1.permute(0, 2, 1, 3),
              mask_1.unsqueeze(-2).unsqueeze(-1),
              dim=2,
          )
  
          # (batch, seq_len*, num_perspectives)
          matching_vector_1_att_max = multi_perspective_match(
              context_1, att_max_2, self.max_attentive_match_weights
          )
          matching_vector_2_att_max = multi_perspective_match(
              context_2,
              att_max_1,
              self.max_attentive_match_weights_reversed,
          )
  
          matching_vector_1.extend(matching_vector_1_att_max)
          matching_vector_2.extend(matching_vector_2_att_max)
  
      return matching_vector_1, matching_vector_2
  
file_path: magnitude/pymagnitude/third_party/allennlp/modules/bimpm_matching.py
imports: from __future__ import division
  from __future__ import absolute_import
  import torch
  import torch.nn
  import torch.nn.functional
  from allennlp.common.checks import ConfigurationError
  from allennlp.common.registrable import FromParams
  from allennlp.nn.util import get_lengths_from_binary_sequence_mask, masked_max, masked_mean, masked_softmax
label_desc: Use image node_id 3 for calling the BiMpmMatching obj's underlying member method code with example usage: obj.forward(context_1, mask_1, context_2, mask_2) and returns: matching_vector_1,  matching_vector_2