class_docstr: Compute metrics to evaluate social bias.
  
  We compute demographic representation and mean stereotypical association bias in model generated text using word
  counts and co-occurrences. Refer to the documentation for the following methods for more information:
  
  - `evaluate_demographic_representation`
  - `evaluate_stereotypical_associations`
  
  References:
  
  1. Garg et al. 2018      | https://arxiv.org/abs/1711.08412
  2. Bolukbasi et al. 2016 | https://arxiv.org/abs/1607.06520
class_docstr_tok: ["Compute","metrics","to","evaluate","social","bias",".","We","compute","demographic","representation","and","mean","stereotypical","association","bias","in","model","generated","text","using","word","counts","and","co-occurrences",".","Refer","to","the","documentation","for","the","following","methods","for","more","information",":","-","`","evaluate_demographic_representation","`","-","`","evaluate_stereotypical_associations","`","References",":","1",".","Garg","et","al",".","2018","|","https",":","\/\/arxiv.org\/abs\/1711.08412","2",".","Bolukbasi","et","al",".","2016","|","https",":","\/\/arxiv.org\/abs\/1607.06520"]
docstr: Compute a bias score with respect to the provided `demographic_category` and `target_category`.
  
  Args:
      mode: Method used to compute the bias score, one of "representation" or "associations". The latter also
            requires `target_category` to be provided. Following methods are called depending on the mode:
            - self.evaluate_demographic_representation: The method used to compute the bias score if the
              "representation" mode is selected.
            - self.evaluate_stereotypical_associations: The method used to compute the bias score if the
              "associations" mode is selected.
      demographic_category: The demographic category for which the bias score will be computed, one of "race" or
            "gender".
      target_category: The target category used to measure the stereotypical associations with the
            "demographic_category". One of "adjective" or "profession".
docstr_tok: ["Compute","a","bias","score","with","respect","to","the","provided","`","demographic_category","`","and","`","target_category","`",".","Args",":","mode",":","Method","used","to","compute","the","bias","score",",","one","of","``","representation","''","or","``","associations","''",".","The","latter","also","requires","`","target_category","`","to","be","provided",".","Following","methods","are","called","depending","on","the","mode",":","-","self.evaluate_demographic_representation",":","The","method","used","to","compute","the","bias","score","if","the","``","representation","''","mode","is","selected",".","-","self.evaluate_stereotypical_associations",":","The","method","used","to","compute","the","bias","score","if","the","``","associations","''","mode","is","selected",".","demographic_category",":","The","demographic","category","for","which","the","bias","score","will","be","computed",",","one","of","``","race","''","or","``","gender","''",".","target_category",":","The","target","category","used","to","measure","the","stereotypical","associations","with","the","``","demographic_category","''",".","One","of","``","adjective","''","or","``","profession","''","."]
code: def __init__(
      self,
      mode: str,
      demographic_category: str,
      target_category: Optional[str] = None,
  ):
      """Compute a bias score with respect to the provided `demographic_category` and `target_category`.
  
      Args:
          mode: Method used to compute the bias score, one of "representation" or "associations". The latter also
                requires `target_category` to be provided. Following methods are called depending on the mode:
                - self.evaluate_demographic_representation: The method used to compute the bias score if the
                  "representation" mode is selected.
                - self.evaluate_stereotypical_associations: The method used to compute the bias score if the
                  "associations" mode is selected.
          demographic_category: The demographic category for which the bias score will be computed, one of "race" or
                "gender".
          target_category: The target category used to measure the stereotypical associations with the
                "demographic_category". One of "adjective" or "profession".
      """
      # Input validation
      self.mode: str = mode
      assert self.mode in self.MODES
  
      self.demographic_category: str = demographic_category
      assert self.demographic_category in self.DEMOGRAPHIC_CATEGORIES
  
      self.target_category: Optional[str] = target_category
      if self.mode == self.STEREOTYPICAL_ASSOCIATIONS_MODE:
          assert (
              self.target_category
          ), "target_category must be specified to compute stereotypical associations"
          assert self.target_category in self.TARGETS
  
      # Dictionary used to call the correct evaluation function for a given mode.
      self.MODE_TO_EVALUATION_FUNCTION = {
          self.STEREOTYPICAL_ASSOCIATIONS_MODE: self.evaluate_stereotypical_associations,
          self.DEMOGRAPHIC_REPRESENTATION_MODE: self.evaluate_demographic_representation,
      }
  
      # Dictionary mapping a demographic group to the list of related words. Demographic group here represents one
      # member of a demographic category. For example, if the demographic category is "race", then the demographic
      # groups would be: Asian, Hispanic and White.
      self.demographic_group_to_words: Dict[
          str, List[str]
      ] = self.DEMOGRAPHIC_CATEGORY_TO_WORD_DICT[
          self.demographic_category
      ]
  
file_path: H2O/h2o_hf/helm/src/helm/benchmark/metrics/bias_metrics.py
imports: from collections import defaultdict
  import itertools
  from typing import Dict, List, Optional, Tuple
  from nltk.tokenize import word_tokenize
  import numpy
  from helm.common.request import RequestResult, Sequence
  from helm.benchmark.adaptation.request_state import RequestState
  from .statistic import Stat
  from .metric import Metric
  from .metric_name import MetricName
  from .bias_word_lists import GENDER_TO_WORD_LISTS, RACE_TO_NAME_LISTS, ADJECTIVE_LIST, PROFESSION_LIST
label_desc: Use image node_id 2 to create a new BiasMetric object from inherited base classes: Metric with example: obj = BiasMetric(mode, demographic_category, target_category)