class_docstr: This class implements Minjoon Seo's `Bidirectional Attention Flow model
  <https://www.semanticscholar.org/paper/Bidirectional-Attention-Flow-for-Machine-Seo-Kembhavi/7586b7cca1deba124af80609327395e613a20e9d>`_
  for answering reading comprehension questions (ICLR 2017).
  
  The basic layout is pretty simple: encode words as a combination of word embeddings and a
  character-level encoder, pass the word representations through a bi-LSTM/GRU, use a matrix of
  attentions to put question information into the passage word representations (this is the only
  part that is at all non-standard), pass this through another few layers of bi-LSTMs/GRUs, and
  do a softmax over span start and span end.
  
  Parameters
  ----------
  vocab : ``Vocabulary``
  text_field_embedder : ``TextFieldEmbedder``
      Used to embed the ``question`` and ``passage`` ``TextFields`` we get as input to the model.
  num_highway_layers : ``int``
      The number of highway layers to use in between embedding the input and passing it through
      the phrase layer.
  phrase_layer : ``Seq2SeqEncoder``
      The encoder (with its own internal stacking) that we will use in between embedding tokens
      and doing the bidirectional attention.
  similarity_function : ``SimilarityFunction``
      The similarity function that we will use when comparing encoded passage and question
      representations.
  modeling_layer : ``Seq2SeqEncoder``
      The encoder (with its own internal stacking) that we will use in between the bidirectional
      attention and predicting span start and end.
  span_end_encoder : ``Seq2SeqEncoder``
      The encoder that we will use to incorporate span start predictions into the passage state
      before predicting span end.
  dropout : ``float``, optional (default=0.2)
      If greater than 0, we will apply dropout with this probability after all encoders (pytorch
      LSTMs do not apply dropout to their last layer).
  mask_lstms : ``bool``, optional (default=True)
      If ``False``, we will skip passing the mask to the LSTM layers.  This gives a ~2x speedup,
      with only a slight performance decrease, if any.  We haven't experimented much with this
      yet, but have confirmed that we still get very similar performance with much faster
      training times.  We still use the mask for all softmaxes, but avoid the shuffling that's
      required when using masking with pytorch LSTMs.
  initializer : ``InitializerApplicator``, optional (default=``InitializerApplicator()``)
      Used to initialize the model parameters.
  regularizer : ``RegularizerApplicator``, optional (default=``None``)
      If provided, will be used to calculate the regularization penalty during training.
class_docstr_tok: ["This","class","implements","Minjoon","Seo","'s","`","Bidirectional","Attention","Flow","model","<","https",":","\/\/www.semanticscholar.org\/paper\/Bidirectional-Attention-Flow-for-Machine-Seo-Kembhavi\/7586b7cca1deba124af80609327395e613a20e9d",">","`","_","for","answering","reading","comprehension","questions","(","ICLR","2017",")",".","The","basic","layout","is","pretty","simple",":","encode","words","as","a","combination","of","word","embeddings","and","a","character-level","encoder",",","pass","the","word","representations","through","a","bi-LSTM\/GRU",",","use","a","matrix","of","attentions","to","put","question","information","into","the","passage","word","representations","(","this","is","the","only","part","that","is","at","all","non-standard",")",",","pass","this","through","another","few","layers","of","bi-LSTMs\/GRUs",",","and","do","a","softmax","over","span","start","and","span","end",".","Parameters","--","--","--","--","--","vocab",":","``","Vocabulary","``","text_field_embedder",":","``","TextFieldEmbedder","``","Used","to","embed","the","``","question","``","and","``","passage","``","``","TextFields","``","we","get","as","input","to","the","model",".","num_highway_layers",":","``","int","``","The","number","of","highway","layers","to","use","in","between","embedding","the","input","and","passing","it","through","the","phrase","layer",".","phrase_layer",":","``","Seq2SeqEncoder","``","The","encoder","(","with","its","own","internal","stacking",")","that","we","will","use","in","between","embedding","tokens","and","doing","the","bidirectional","attention",".","similarity_function",":","``","SimilarityFunction","``","The","similarity","function","that","we","will","use","when","comparing","encoded","passage","and","question","representations",".","modeling_layer",":","``","Seq2SeqEncoder","``","The","encoder","(","with","its","own","internal","stacking",")","that","we","will","use","in","between","the","bidirectional","attention","and","predicting","span","start","and","end",".","span_end_encoder",":","``","Seq2SeqEncoder","``","The","encoder","that","we","will","use","to","incorporate","span","start","predictions","into","the","passage","state","before","predicting","span","end",".","dropout",":","``","float","``",",","optional","(","default=0.2",")","If","greater","than","0",",","we","will","apply","dropout","with","this","probability","after","all","encoders","(","pytorch","LSTMs","do","not","apply","dropout","to","their","last","layer",")",".","mask_lstms",":","``","bool","``",",","optional","(","default=True",")","If","``","False","``",",","we","will","skip","passing","the","mask","to","the","LSTM","layers",".","This","gives","a","~2x","speedup",",","with","only","a","slight","performance","decrease",",","if","any",".","We","have","n't","experimented","much","with","this","yet",",","but","have","confirmed","that","we","still","get","very","similar","performance","with","much","faster","training","times",".","We","still","use","the","mask","for","all","softmaxes",",","but","avoid","the","shuffling","that's","required","when","using","masking","with","pytorch","LSTMs",".","initializer",":","``","InitializerApplicator","``",",","optional","(","default=","``","InitializerApplicator","(",")","``",")","Used","to","initialize","the","model","parameters",".","regularizer",":","``","RegularizerApplicator","``",",","optional","(","default=","``","None","``",")","If","provided",",","will","be","used","to","calculate","the","regularization","penalty","during","training","."]
code: def __init__(
      self,
      vocab,
      text_field_embedder,
      num_highway_layers,
      phrase_layer,
      similarity_function,
      modeling_layer,
      span_end_encoder,
      dropout=0.2,
      mask_lstms=True,
      initializer=InitializerApplicator(),
      regularizer=None,
  ):
      super(BidirectionalAttentionFlow, self).__init__(
          vocab, regularizer
      )
  
      self._text_field_embedder = text_field_embedder
      self._highway_layer = TimeDistributed(
          Highway(
              text_field_embedder.get_output_dim(), num_highway_layers
          )
      )
      self._phrase_layer = phrase_layer
      self._matrix_attention = LegacyMatrixAttention(
          similarity_function
      )
      self._modeling_layer = modeling_layer
      self._span_end_encoder = span_end_encoder
  
      encoding_dim = phrase_layer.get_output_dim()
      modeling_dim = modeling_layer.get_output_dim()
      span_start_input_dim = encoding_dim * 4 + modeling_dim
      self._span_start_predictor = TimeDistributed(
          torch.nn.Linear(span_start_input_dim, 1)
      )
  
      span_end_encoding_dim = span_end_encoder.get_output_dim()
      span_end_input_dim = encoding_dim * 4 + span_end_encoding_dim
      self._span_end_predictor = TimeDistributed(
          torch.nn.Linear(span_end_input_dim, 1)
      )
  
      # Bidaf has lots of layer dimensions which need to match up - these aren't necessarily
      # obvious from the configuration files, so we check here.
      check_dimensions_match(
          modeling_layer.get_input_dim(),
          4 * encoding_dim,
          "modeling layer input dim",
          "4 * encoding dim",
      )
      check_dimensions_match(
          text_field_embedder.get_output_dim(),
          phrase_layer.get_input_dim(),
          "text field embedder output dim",
          "phrase layer input dim",
      )
      check_dimensions_match(
          span_end_encoder.get_input_dim(),
          4 * encoding_dim + 3 * modeling_dim,
          "span end encoder input dim",
          "4 * encoding dim + 3 * modeling dim",
      )
  
      self._span_start_accuracy = CategoricalAccuracy()
      self._span_end_accuracy = CategoricalAccuracy()
      self._span_accuracy = BooleanAccuracy()
      self._squad_metrics = SquadEmAndF1()
      if dropout > 0:
          self._dropout = torch.nn.Dropout(p=dropout)
      else:
          self._dropout = lambda x: x
      self._mask_lstms = mask_lstms
  
      initializer(self)
  
file_path: magnitude/pymagnitude/third_party/allennlp/models/reading_comprehension/bidaf.py
imports: from __future__ import absolute_import
  import logging
  import torch
  from torch.nn.functional import nll_loss
  from allennlp.common.checks import check_dimensions_match
  from allennlp.data import Vocabulary
  from allennlp.models.model import Model
  from allennlp.modules import Highway
  from allennlp.modules import Seq2SeqEncoder, SimilarityFunction, TimeDistributed, TextFieldEmbedder
  from allennlp.modules.matrix_attention.legacy_matrix_attention import LegacyMatrixAttention
  from allennlp.nn import util, InitializerApplicator, RegularizerApplicator
  from allennlp.training.metrics import BooleanAccuracy, CategoricalAccuracy, SquadEmAndF1
label_desc: Use image node_id 1 to create a new BidirectionalAttentionFlow object from inherited base classes: Model with example: obj = BidirectionalAttentionFlow(vocab, text_field_embedder, num_highway_layers, phrase_layer, similarity_function, modeling_layer, span_end_encoder, dropout, mask_lstms, initializer, regularizer)