5. Examining approaches to building a books data 
commons 
There are many possible permutations for building a books data commons. To structure our 
exploration, we focused on two particular tracks, discussed below. We chose these tracks 
mindful of the above legal issues, and because there are already existence proofs that help 
to illuminate tradeoffs, challenges and potential paths forward for each.  
5a. Public domain and permissively licensed books 
Existing Project Example : The Pile v2 27
In 2020, the nonproﬁt research group EleutherAI constructed and released The Pile — a large, 
diverse, open dataset for AI training. EleutherAI developed it not only to support their own 
training of LLMs, but also to lower the barriers for others.  28
Along with data drawn from the web at large, The Pile included books from three datasets. 
The ﬁrst dataset was the Books3 corpus referenced at the outset of this paper. The second 
and third books datasets were smaller: BookCorpus2, which is a collection of 17,868 books 
by otherwise unpublished authors; and a 28,752 books in the public domain and published 
prior to 1919, drawn from a volunteer effort to digitize public domain works called Project 
Gutenberg.  
As the awareness about The Pile dataset grew, certain rightsholders began sending copyright 
notices to have the dataset taken down from various websites. 
Despite the takedown requests, the importance of books to EleutherAI and the broader 
community’s AI research remained. In hoping to forge a path forward EleutherAI announced 
in 2024 that they would create a new version of the dataset, which they will call The Pile v2.  29
Among other things, v2 would “have many more books than the original Pile had, for 
example, and more diverse representation of non-academic non- ﬁction domains.” At the 
same time, it would only seek to include public domain books and permissively licensed 
content. As before, this corpus focuses on English language books. 
 This is an illustrative example, and there are also other projects of this ilk. For instance, see the 27
Common Corpus project, which includes an array of public domain books from a number of countries, 
at https://huggingface.co/blog/Pclanglais/common-corpus; see also https://huggingface.co/datasets/
storytracer/internet_archive_books_en (“This dataset contains more than 650,000 English public domain 
books (~ 61 billion words) which were digitized by the Internet Archive and cataloged as part of the 
Open Library project.”)
 See Gao et al, supra note 8.28
 Goldman, Sharon. “One of the World’s Largest AI Training Datasets Is About to Get Bigger and 29
“Substantially Better.” VentureBeat, 11 Jan. 2024, venturebeat.com/ai/one-of-the-worlds-largest-ai-
training-datasets-is-about-to-get-bigger-and-substantially-better/. Accessed 20 Mar. 2024.
Towards a Books Data Commons for AI Training 12