text: 6.1 Coherence in the Eye of the Beholder 
  Where traditional n-gram LMs [117] can only model relatively local 
  dependencies, predicting each word given the preceding sequence 
  of N words (usually 5 or fewer), the Transformer LMs capture 
  much larger windows and can produce text that is seemingly not 
  only fluent but also coherent even over paragraphs. For example, 
  McGuffie and Newhouse [80] prompted GPT-3 with the text in 
  bold in Figure 1, and it produced the rest of the text, including the 
  Q&A format.21 This example illustrates GPT-3’s ability to produce 
  coherent and on-topic text; the topic is connected to McGuffie and 
  Newhouse’s study of GPT-3 in the context of extremism, discussed 
  below. 
  
  We say seemingly coherent because coherence is in fact in the 
  eye of the beholder. Our human understanding of coherence de- 
  rives from our ability to recognize interlocutors’ beliefs [30, 31] and 
  intentions [23, 33] within context [32]. That is, human language use 
  
  21This is just the first part of the response that McGuffie and Newhouse show. GPT-3 
  continues for two more question answer pairs with similar coherence. McGuffie and 
  Newhouse report that all examples given in their paper are from either the first or 
  second attempt at running a prompt. 