3. Why Books are Important to Training AI   
Despite the proliferation of online content and some speculating that books would simply die 
out with the advent of the Internet,  books remain a critical vehicle for disseminating 9
knowledge. The more scientists study how books can impact people, the less surprising this 
is. Our brains have been shown to interact with longform books in meaningful ways: we 
develop bigger vocabularies when we read books; we develop more empathy when we read 
literary ﬁction; and connectivity between different regions of our brain increases when we 
read.   10
In that light, it might be unsurprising that books are important for training AI models. A 
broadly accessible books dataset could be useful not only for building LLMs, but also for 
many other types of AI research and development.  
Performance and Quality 
The performance and versatility of an AI model can signi ﬁcantly depend on whether the 
training corpus includes books or not. Books are uniquely valuable for AI training due to 
several characteristics. 
• Length: Books tend to represent longer-form content, and ﬁction books, in particular, 
represent long-form narrative. An AI trained on this longer-form, narrative type of 
content is able to make connections over a longer context, so instead of putting 
words together to form a single sentence, the AI becomes more able to string 
concepts together into a coherent whole; even after a book is divided into many 
“chunks” before the process of tokenization, that will still provide long stretches of 
text that are longer than the average web page. While Web documents, for instance, 
tend to be longer than a single sentence, they are not typically hundreds of pages long 
like a book. 
• Quality: The qualities of the training data impact the outputs a tool can produce. 
Consider an LLM trained on gibberish; it can learn the patterns of that gibberish and, 
in turn, produce related gibberish, but will not be very useful for writing an argument 
or a story, for instance. In contrast, training an LLM on books with well-constructed 
arguments or crafted stories could serve those purposes. While “well-constructed” 
and “crafted” are necessarily subjective, the traditional role of editors and the 
publishing process can provide a useful indicator for the quality of writing inside of 
books. What’s more, metadata for books — information such as the title, author and 
year of publication — is often more comprehensive than metadata for information 
 “the novel, too, as we know it, has come to its end” — “The End of Books.” Archive.nytimes.com, 21 June 9
1992, archive.nytimes.com/www.nytimes.com/books/98/09/27/specials/coover-end.html. Accessed 
27 Aug. 2021.
 Stanborough, Rebecca Joy. “Beneﬁts of Reading Books: For Your Physical and Mental Health.” 10
Healthline, 15 Oct. 2019, www.healthline.com/health/beneﬁts-of-reading-books#prevents-cognitive-
decline.
Towards a Books Data Commons for AI Training 5