text: Just as environmental impact scales with model size, so does 
  the difficulty of understanding what is in the training data. In §4, 
  we discuss how large datasets based on texts from the Internet 
  overrepresent hegemonic viewpoints and encode biases potentially 
  damaging to marginalized populations. In collecting ever larger 
  datasets we risk incurring documentation debt. We recommend 
  mitigating these risks by budgeting for curation and documentation 
  at the start of a project and only creating datasets as large as can 
  be sufficiently documented. 
  
  As argued by Bender and Koller [14], it is important to under- 
  stand the limitations of LMs and put their success in context. This 
  not only helps reduce hype which can mislead the public and re- 
  searchers themselves regarding the capabilities of these LMs, but 
  might encourage new research directions that do not necessarily 
  depend on having larger LMs. As we discuss in §5, LMs are not 
  performing natural language understanding (NLU), and only have 
  success in tasks that can be approached by manipulating linguis- 
  tic form [14]. Focusing on state-of-the-art results on leaderboards 
  without encouraging deeper understanding of the mechanism by 
  which they are achieved can cause misleading results as shown 