text: FAccT ’21, March 3–10, 2021, Virtual Event, Canada 
  
  means of measuring the toxicity of text generated by LMs. For 
  example, the Perspective API model has been found to associate 
  higher levels of toxicity with sentences containing identity markers 
  for marginalized groups or even specific names [61, 103]. 
  
  Second, auditing an LM for biases requires an a priori under- 
  standing of what social categories might be salient. The works cited 
  above generally start from US protected attributes such as race and 
  gender (as understood within the US). But, of course, protected 
  attributes aren’t the only identity characteristics that can be subject 
  to bias or discrimination, and the salient identity characteristics 
  and expressions of bias are also culture-bound [46, 116]. Thus, com- 
  ponents like toxicity classifiers would need culturally appropriate 
  training data for each context of audit, and even still we may miss 
  marginalized identities if we don’t know what to audit for. 