text: Tokens do not typically represent words, but instead often represent subword tokens. For 
  example the word “incompetence” may be broken into three tokens: “in-,” “competent,” and “- 
  ence.” This approach to tokenization enables representation of grammar and word variations, 
  effectively allowing a high degree of language generalizability. 7 
  
  In recent years, LLM research has successfully been able to scale up models by pre-training 
  on a large number of tokens. In turn, this has allowed a higher degree of language 
  generalizability in the resulting model. For example, OpenAI’s ChatGPT trained on hundreds 
  of billions of tokens, allowing it to model language in a very general way. The resulting 
  models an then be ﬁne-tuned for speciﬁc tasks using training data representing a particular 
  corpus, such as software code. 8 