class_docstr: Scenario for single document text summarization.
  Currently, supports the following datasets:
  1. BillSum (https://aclanthology.org/D19-5406/)
  2. MultiLexSum (https://arxiv.org/abs/2206.10883)
  3. EurLexSum (https://arxiv.org/abs/2210.13448)
  
  Task prompt structure
  
      Summarize the given document.
      Document: {tok_1 ... tok_n}
      Summary: {tok_1 ... tok_m}
  
  Example from MultiLexSum dataset (Short to Tiny)
  
      Document: {This case is about an apprenticeship test that had a disparate impact
                  on Black apprenticeship applicants. The Equal Employment Opportunity
                  Commission (EEOC) filed this lawsuit on December 27, 2004, in U.S.
                  District Court for the Southern District of Ohio. Filing on behalf
                  of thirteen Black individuals and a class of similarly situated Black
                  apprenticeship test takers, the EEOC alleged that the individuals’
                  employer, the Ford Motor Company, as well as their union, the United
                  Automobile, Aerospace, and Agricultural implement workers of America
                  (the “UAW”), and the Ford-UAW Joint Apprenticeship Committee, violated
                  Title VII of the Civil Rights Act, 42 U.S.C. § 1981, and Michigan state
                  anti-discrimination law. The EEOC sought injunctive relief and damages
                  for the Black apprenticeship applicants. The individuals also brought a
                  separate class action against Ford and the UAW, and the cases were
                  consolidated. In June 2005, both cases were resolved via a class
                  settlement agreement. Ford agreed to pay $8.55 million and to implement
                  a new selection process for its apprenticeship programs, and the court
                  ordered Ford to cover attorneys’ fees and expenses. This case is closed.}
      Summary: {2005 class action settlement resulted in Ford paying $8.55m to redesign
                  its selection process for apprenticeship programs to address the
                  previous process’s disparate impact on Black applicants.}
class_docstr_tok: ["Scenario","for","single","document","text","summarization",".","Currently",",","supports","the","following","datasets",":","1",".","BillSum","(","https",":","\/\/aclanthology.org\/D19-5406\/",")","2",".","MultiLexSum","(","https",":","\/\/arxiv.org\/abs\/2206.10883",")","3",".","EurLexSum","(","https",":","\/\/arxiv.org\/abs\/2210.13448",")","Task","prompt","structure","Summarize","the","given","document",".","Document",":","{","tok_1","...","tok_n","}","Summary",":","{","tok_1","...","tok_m","}","Example","from","MultiLexSum","dataset","(","Short","to","Tiny",")","Document",":","{","This","case","is","about","an","apprenticeship","test","that","had","a","disparate","impact","on","Black","apprenticeship","applicants",".","The","Equal","Employment","Opportunity","Commission","(","EEOC",")","filed","this","lawsuit","on","December","27",",","2004",",","in","U.S.","District","Court","for","the","Southern","District","of","Ohio",".","Filing","on","behalf","of","thirteen","Black","individuals","and","a","class","of","similarly","situated","Black","apprenticeship","test","takers",",","the","EEOC","alleged","that","the","individuals","\u2019","employer",",","the","Ford","Motor","Company",",","as","well","as","their","union",",","the","United","Automobile",",","Aerospace",",","and","Agricultural","implement","workers","of","America","(","the","\u201c","UAW","\u201d",")",",","and","the","Ford-UAW","Joint","Apprenticeship","Committee",",","violated","Title","VII","of","the","Civil","Rights","Act",",","42","U.S.C",".","\u00a7","1981",",","and","Michigan","state","anti-discrimination","law",".","The","EEOC","sought","injunctive","relief","and","damages","for","the","Black","apprenticeship","applicants",".","The","individuals","also","brought","a","separate","class","action","against","Ford","and","the","UAW",",","and","the","cases","were","consolidated",".","In","June","2005",",","both","cases","were","resolved","via","a","class","settlement","agreement",".","Ford","agreed","to","pay","$","8.55","million","and","to","implement","a","new","selection","process","for","its","apprenticeship","programs",",","and","the","court","ordered","Ford","to","cover","attorneys","\u2019","fees","and","expenses",".","This","case","is","closed",".","}","Summary",":","{","2005","class","action","settlement","resulted","in","Ford","paying","$","8.55m","to","redesign","its","selection","process","for","apprenticeship","programs","to","address","the","previous","process","\u2019","s","disparate","impact","on","Black","applicants",".","}"]
docstr: Initializes summarization scenario.
  Args:
      dataset_name: String identifier for dataset. Currently
                    supported options ["BillSum", "MultiLexSum", "EurLexSum"].
      sampling_min_length: Int indicating minimum length (num whitespace-separated tokens) for training
                           documents. Training examples smaller than
                           sampling_min_length will be filtered out.
                           Useful for preventing the adapter from sampling
                           really small documents.
      sampling_max_length: Int indicating maximum length (num whitespace-separated tokens) for training
                           documents. Training examples larger than
                           sampling_max_length will be filtered out.
                           Useful for preventing the adapter from
                           sampling really large documents.
      doc_max_length: Int indicating the maximum length (num whitespace-separated tokens) to truncate
                      documents. Documents in all splits will be
                      truncated to doc_max_length tokens.
                      NOTE: Currently uses whitespace tokenization.
docstr_tok: ["Initializes","summarization","scenario",".","Args",":","dataset_name",":","String","identifier","for","dataset",".","Currently","supported","options","[","``","BillSum","''",",","``","MultiLexSum","''",",","``","EurLexSum","''","]",".","sampling_min_length",":","Int","indicating","minimum","length","(","num","whitespace-separated","tokens",")","for","training","documents",".","Training","examples","smaller","than","sampling_min_length","will","be","filtered","out",".","Useful","for","preventing","the","adapter","from","sampling","really","small","documents",".","sampling_max_length",":","Int","indicating","maximum","length","(","num","whitespace-separated","tokens",")","for","training","documents",".","Training","examples","larger","than","sampling_max_length","will","be","filtered","out",".","Useful","for","preventing","the","adapter","from","sampling","really","large","documents",".","doc_max_length",":","Int","indicating","the","maximum","length","(","num","whitespace-separated","tokens",")","to","truncate","documents",".","Documents","in","all","splits","will","be","truncated","to","doc_max_length","tokens",".","NOTE",":","Currently","uses","whitespace","tokenization","."]
code: def __init__(
      self,
      dataset_name: str,
      sampling_min_length: Optional[int] = None,
      sampling_max_length: Optional[int] = None,
      doc_max_length: Optional[int] = None,
  ):
      """
      Initializes summarization scenario.
      Args:
          dataset_name: String identifier for dataset. Currently
                        supported options ["BillSum", "MultiLexSum", "EurLexSum"].
          sampling_min_length: Int indicating minimum length (num whitespace-separated tokens) for training
                               documents. Training examples smaller than
                               sampling_min_length will be filtered out.
                               Useful for preventing the adapter from sampling
                               really small documents.
          sampling_max_length: Int indicating maximum length (num whitespace-separated tokens) for training
                               documents. Training examples larger than
                               sampling_max_length will be filtered out.
                               Useful for preventing the adapter from
                               sampling really large documents.
          doc_max_length: Int indicating the maximum length (num whitespace-separated tokens) to truncate
                          documents. Documents in all splits will be
                          truncated to doc_max_length tokens.
                          NOTE: Currently uses whitespace tokenization.
      """
      super().__init__()
      if dataset_name not in ["BillSum", "MultiLexSum", "EurLexSum"]:
          raise Exception(f"Unknown dataset_name: {dataset_name}")
      self.dataset_name = dataset_name
      self.sampling_min_length = sampling_min_length
      self.sampling_max_length = sampling_max_length
      self.doc_max_length = doc_max_length
  
file_path: H2O/h2o_hf/helm/src/helm/benchmark/scenarios/legal_summarization_scenario.py
imports: from pathlib import Path
  from typing import List, Optional, Any
  import datasets
  from datasets import load_dataset
  from .scenario import Scenario, Instance, Reference, TRAIN_SPLIT, VALID_SPLIT, TEST_SPLIT, CORRECT_TAG, Input, Output
label_desc: Use image node_id 1 to create a new LegalSummarizationScenario object from inherited base classes: Scenario with example: obj = LegalSummarizationScenario(dataset_name, sampling_min_length, sampling_max_length, doc_max_length)