Implications of the The Overall Approach 
Stepping back from The Pile v2 speci ﬁcally, or any particular existing collection of books or 
dataset built on their basis, we want to understand the implications of relying on public 
domain works and expressly licensed works in building a books commons. 
The beneﬁts are relatively straightforward. Both categories, by de ﬁnition come with express 
permission to use the books in AI training. The cost of acquiring the books for this use may 
be effectively zero or close to it, when considering public domain and “openly” licensed 
books that allow redistribution and that have already been digitized.  
But this approach comes with some clear limitations. First, as noted above, for many books 
in the public domain, their status as such is not always clear. And with respect to 
permissively licensed books, it is not always clear whether and how to comply with the 
license obligations in this context.  
Setting aside those challenges, the simple fact is that relying on public domain and existing 
permissively licensed books would limit the quantity and diversity of data available for 
training, impacting performance along different dimensions. Only a small fraction of books 
ever published fall into this category, and the corpus of books in this category is likely to be 
skewed heavily towards older public domain books. This skew would, in turn, impact the 
content available for AI training.  For instance, relying on books from before 1929 would not 30
only incorporate outdated language patterns, but also a range of biases and misconceptions 
about race and gender, among other things. Efforts could be made to get people to 
permissively license more material — a book drive for permissive licensing, so to speak; this 
approach would still not encompass most books, at least when it comes to past works.  31
5b. Limitations & Exceptions  
Existing Project Example: HathiTrust Research Center (HTRC) 
The HathiTrust Research Center provides researchers with the ability to perform 
computational analysis across millions of books. While it is not suited speci ﬁcally for AI 
training, it is an existence proof for what such a resource might look like.  
 For instance, AI researchers note that the recently released Common Corpus dataset is an “invaluable 30
resource” but “comes with limitations. A lot of public domain data is antiquated—in the US, for example, 
copyright protection usually lasts over seventy years from the death of the author—so this type of 
dataset won’t be able to ground an AI model in current affairs or, say, how to spin up a blog post using 
current slang” and the “dataset is tiny.” Thus, while it is possible to train an AI model on the data, those 
models will have more limited utility on some dimensions than current frontier models trained on a 
broader array of data. See Knibbs, Kate, Here’s Proof You Can Train an AI Model Without Slurping 
Copyrighted Content | WIRED. (2024, March 20), at https://www.wired.com/story/proof-you-can-train-ai-
without-slurping-copyrighted-content/.
 Our workshop discussion did note that some widely available datasets for AI training have also 31
pursued more direct licensing agreements. For instance, the SILO LLM was created by working with 
scientiﬁc journal publishers to make works available for both download and AI training. While this might 
be viable in the context of particular, narrow classes of works, the barriers to eﬃcient licensing 
mentioned above would remain a problem for any broader efforts. See Min, Sewon, et al. “SILO 
Language Models: Isolating Legal Risk in a Nonparametric Datastore.” ArXiv (Cornell University), 8 Aug. 
2023, https://doi.org/10.48550/arxiv.2308.04430. Accessed 14 Dec. 2023.
Towards a Books Data Commons for AI Training 13