class_docstr: From https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification/data:
  At the end of 2017 the Civil Comments platform shut down and chose make their ~2m public comments from their
  platform available in a lasting open archive so that researchers could understand and improve civility in
  online conversations for years to come. Jigsaw sponsored this effort and extended annotation of this data
  by human raters for various toxic conversational attributes.
  
  We used the WILDS version of CivilComments (see https://arxiv.org/abs/2012.07421 on how the original
  dataset was modified). The eight demographics for the WILDS version of the dataset were:
  
  - Male
  - Female
  - LGBTQ
  - Christian
  - Muslim
  - Other religions
  - Black
  - White
  
  Example
  
  ```
  Blame men.  There's always an excuse to blame men for the failures of women.
  Question: Is the passage above toxic?"
  
  True
  ```
  
  References
  
      @inproceedings{wilds2021,
          title = {{WILDS}: A Benchmark of in-the-Wild Distribution Shifts},
          author = {Pang Wei Koh and Shiori Sagawa and Henrik Marklund and Sang Michael Xie and Marvin Zhang and
          Akshay Balsubramani and Weihua Hu and Michihiro Yasunaga and Richard Lanas Phillips and Irena Gao and
          Tony Lee and Etienne David and Ian Stavness and Wei Guo and Berton A. Earnshaw and Imran S. Haque and
          Sara Beery and Jure Leskovec and Anshul Kundaje and Emma Pierson and Sergey Levine and Chelsea Finn
          and Percy Liang},
          booktitle = {International Conference on Machine Learning (ICML)},
          year = {2021}
      }
  
      @inproceedings{borkan2019nuanced,
          title={Nuanced metrics for measuring unintended bias with real data for text classification},
          author={Borkan, Daniel and Dixon, Lucas and Sorensen, Jeffrey and Thain, Nithum and Vasserman, Lucy},
          booktitle={Companion Proceedings of The 2019 World Wide Web Conference},
          pages={491--500},
          year={2019}
      }
code: def __init__(self, demographic: str):
      super().__init__()
      assert (
          demographic == CivilCommentsScenario.ALL_DEMOGRAPHICS
          or demographic in CivilCommentsScenario.DEMOGRAPHICS
      ), f"Invalid demographic: {demographic}"
      self.demographic: str = demographic
  
file_path: H2O/h2o_hf/helm/src/helm/benchmark/scenarios/civil_comments_scenario.py
imports: import os
  from typing import Dict, List, Set
  import pandas
  from helm.common.general import ensure_file_downloaded
  from .scenario import CORRECT_TAG, TEST_SPLIT, TRAIN_SPLIT, VALID_SPLIT, Instance, PassageQuestionInput, Reference, Scenario, Output
label_desc: Use image node_id 1 to create a new CivilCommentsScenario object from inherited base classes: Scenario with example: obj = CivilCommentsScenario(demographic)