class_docstr: Contrast Sets are from this paper (currently supported for the BoolQ and IMDB scenarios):
  https://arxiv.org/abs/2004.02709
  
  Original repository can be found at:
  https://github.com/allenai/contrast-sets
  
  An example instance of a perturbation for the BoolQ dataset:
  
  ```
  The Fate of the Furious premiered in Berlin on April 4, 2017, and was theatrically released in the United States on
  April 14, 2017, playing in 3D, IMAX 3D and 4DX internationally. . . A spinoff film starring Johnson and Statham’s
  characters is scheduled for release in August 2019, while the ninth and tenth films are scheduled for releases on
  the years 2020 and 2021.
  question: is “Fate and the Furious” the last movie?
  answer: no
  
  perturbed question: is “Fate and the Furious” the first of multiple movies?
  perturbed answer: yes
  perturbation strategy: adjective change.
  ```
  
  An example instance of a perturbation for the IMDB dataset (from the original paper):
  
  ```
  Original instance: Hardly one to be faulted for his ambition or his vision, it is genuinely unexpected, then, to see
  all Park’s effort add up to so very little. . . .  The premise is promising, gags are copious and offbeat humour
  abounds but it all fails miserably to create any meaningful connection with the audience.
  Sentiment: negative
  
  Perturbed instance: Hardly one to be faulted for his ambition or his vision, here we
  see all Park’s effort come to fruition. . . . The premise is perfect, gags are
  hilarious and offbeat humour abounds, and it creates a deep connection with the
  audience.
  Sentiment: positive
  ```
class_docstr_tok: ["Contrast","Sets","are","from","this","paper","(","currently","supported","for","the","BoolQ","and","IMDB","scenarios",")",":","https",":","\/\/arxiv.org\/abs\/2004.02709","Original","repository","can","be","found","at",":","https",":","\/\/github.com\/allenai\/contrast-sets","An","example","instance","of","a","perturbation","for","the","BoolQ","dataset",":","``","`","The","Fate","of","the","Furious","premiered","in","Berlin","on","April","4",",","2017",",","and","was","theatrically","released","in","the","United","States","on","April","14",",","2017",",","playing","in","3D",",","IMAX","3D","and","4DX","internationally",".",".",".","A","spinoff","film","starring","Johnson","and","Statham","\u2019","s","characters","is","scheduled","for","release","in","August","2019",",","while","the","ninth","and","tenth","films","are","scheduled","for","releases","on","the","years","2020","and","2021.","question",":","is","\u201c","Fate","and","the","Furious","\u201d","the","last","movie","?","answer",":","no","perturbed","question",":","is","\u201c","Fate","and","the","Furious","\u201d","the","first","of","multiple","movies","?","perturbed","answer",":","yes","perturbation","strategy",":","adjective","change",".","``","`","An","example","instance","of","a","perturbation","for","the","IMDB","dataset","(","from","the","original","paper",")",":","``","`","Original","instance",":","Hardly","one","to","be","faulted","for","his","ambition","or","his","vision",",","it","is","genuinely","unexpected",",","then",",","to","see","all","Park","\u2019","s","effort","add","up","to","so","very","little",".",".",".",".","The","premise","is","promising",",","gags","are","copious","and","offbeat","humour","abounds","but","it","all","fails","miserably","to","create","any","meaningful","connection","with","the","audience",".","Sentiment",":","negative","Perturbed","instance",":","Hardly","one","to","be","faulted","for","his","ambition","or","his","vision",",","here","we","see","all","Park","\u2019","s","effort","come","to","fruition",".",".",".",".","The","premise","is","perfect",",","gags","are","hilarious","and","offbeat","humour","abounds",",","and","it","creates","a","deep","connection","with","the","audience",".","Sentiment",":","positive","``","`"]
code: def perturb(
      self, text: str, rng: Random
  ) -> str:  # we need this since parent method is abstract
      pass
  
file_path: H2O/h2o_hf/helm/src/helm/benchmark/augmentations/contrast_sets_perturbation.py
imports: from dataclasses import replace
  from random import Random
  from typing import List, Optional
  from helm.benchmark.scenarios.scenario import Instance, Reference, Input
  from .perturbation_description import PerturbationDescription
  from .perturbation import Perturbation
label_desc: Use image node_id 4 for calling the ContrastSetsPerturbation obj's underlying member method code with example usage: obj.perturb(text, rng) without return types