class_docstr: Natural language inference scores measure the effect biased associations have on decisions
  made downstream, given neutrally-constructed pairs of sentences differing only in the subject.
  
  1. Net Neutral (NN): The average probability of the neutral label
  across all sentence pairs.
  
  2. Fraction Neutral (FN): The fraction of sentence pairs predicted neutral.
  
  3. Threshold:tau (T:tau): A parameterized measure that reports the fraction
  of examples whose probability of neutral is above tau.
  
  # Parameters
  
  neutral_label : `int`, optional (default=`2`)
      The discrete integer label corresponding to a neutral entailment prediction.
  taus : `List[float]`, optional (default=`[0.5, 0.7]`)
      All the taus for which to compute Threshold:tau.
  
  Based on: Dev, S., Li, T., Phillips, J.M., & Srikumar, V. (2020). [On Measuring and Mitigating
  Biased Inferences of Word Embeddings](https://api.semanticscholar.org/CorpusID:201670701).
  ArXiv, abs/1908.09369.
class_docstr_tok: ["Natural","language","inference","scores","measure","the","effect","biased","associations","have","on","decisions","made","downstream",",","given","neutrally-constructed","pairs","of","sentences","differing","only","in","the","subject",".","1",".","Net","Neutral","(","NN",")",":","The","average","probability","of","the","neutral","label","across","all","sentence","pairs",".","2",".","Fraction","Neutral","(","FN",")",":","The","fraction","of","sentence","pairs","predicted","neutral",".","3",".","Threshold",":","tau","(","T",":","tau",")",":","A","parameterized","measure","that","reports","the","fraction","of","examples","whose","probability","of","neutral","is","above","tau",".","#","Parameters","neutral_label",":","`","int","`",",","optional","(","default=","`","2","`",")","The","discrete","integer","label","corresponding","to","a","neutral","entailment","prediction",".","taus",":","`","List","[","float","]","`",",","optional","(","default=","`","[","0.5",",","0.7","]","`",")","All","the","taus","for","which","to","compute","Threshold",":","tau",".","Based","on",":","Dev",",","S.",",","Li",",","T.",",","Phillips",",","J.M.",",","&","Srikumar",",","V.","(","2020",")",".","[","On","Measuring","and","Mitigating","Biased","Inferences","of","Word","Embeddings","]","(","https",":","\/\/api.semanticscholar.org\/CorpusID:201670701",")",".","ArXiv",",","abs\/1908.09369","."]
code: def __init__(
      self, neutral_label: int = 2, taus: List[float] = [0.5, 0.7]
  ):
      self.neutral_label = neutral_label
      self.taus = taus
  
      self._nli_probs_sum = 0.0
      self._num_neutral_predictions = 0.0
      self._num_neutral_above_taus = {tau: 0.0 for tau in taus}
      self._total_predictions = 0
  
imports: from typing import Optional, Dict, Union, List
  import torch
  import torch.distributed
  from allennlp.common.util import is_distributed
  from allennlp.common.checks import ConfigurationError
  from allennlp.nn.util import dist_reduce_sum
  from allennlp.training.metrics.metric import Metric
label_desc: Use image node_id 1 to create a new NaturalLanguageInference object from inherited base classes: Metric with example: obj = NaturalLanguageInference(neutral_label, taus)