7. Conclusion 
This paper is a snapshot of an idea that is as underexplored as it is rooted in decades of 
existing work. The concept of mass digitization of books, including to support text and data 
mining, of which AI is a subset, is not new. But AI training is newly of the zeitgeist, and its 
transformative use makes questions about how we digitize, preserve, and make accessible 
knowledge and cultural heritage salient in a distinct way. 
As such, efforts to build a books data commons need not start from scratch; there is much 
to glean from studying and engaging existing and previous efforts. Those learnings might 
inform substantive decisions about how to build a books data commons for AI training. For 
instance, looking at the design decisions of HathiTrust may inform how the technical 
infrastructure and data management practices for AI training might be designed, as well as 
how to address challenges to building a comprehensive, diverse, and useful corpus. In 
addition, learnings might inform the process by which we get to a books data commons — 
for example, illustrating ways to attend to the interests of those likely to be impacted by the 
dataset’s development.   41
While this paper does not prescribe a particular path forward, we do think ﬁnding a path (or 
paths) to extend access to books for AI training is critical. In the status quo, large swaths of 
knowledge contained in books are effectively locked up and inaccessible to most everyone. 
Google is an exception — it can reap the bene ﬁts of their 40 million books dataset for 
research, development, and deployment of AI models. Large, well-resourced entities could 
theoretically try to replicate Google’s digitization efforts, although it would be incredibly 
expensive, impractical, and largely duplicative for each entity to individually pursue their own 
efforts. Even then, it isn’t clear how everyone else — independent researchers, entrepreneurs, 
and smaller entities — will have access. The controversy around the Books3 dataset 
discussed at the outset should not, then, be an argument in favor of preserving the status 
quo. Instead, it should highlight the urgency of building a books data commons to support an 
AI ecosystem that provides broad beneﬁts beyond the privileged few. 
 For other existing and past examples, one might look to the work of Europeana, https://41
www.europeana.eu/en, as well as the mountain of commentary on the failed class action settlement 
between Google, the Authors Guild, and the Association of American Publishers — see e.g. the excellent 
collection of court ﬁlings created by James Grimmelmann and colleagues (now archived at the Internet 
Archive) — https://web.archive.org/web/20140425012526/http://thepublicindex.org/. The Settlement 
expressly would have set up a “Research Corpus” for non-consumptive research. HathiTrust created a 
Research Center, with the intention of becoming one of the hosts for the “Research Corpus.” The 
Settlement was criticized and was ultimately rejected by the district court for both substantive reasons 
(that is, what the settlement would speciﬁcally do) and procedural (in the sense of violating class-action 
law, but also in a broader sense of representing a “backroom deal” without suﬃcient participation from 
impacted interests). The Research Corpus was not a core locus of critique, though it did receive concern 
in terms of providing too much control to Google, for example. Our purpose in mentioning this is not to 
relitigate the issue, but rather to call out that design decisions of this sort have been considered in the 
past.
Towards a Books Data Commons for AI Training 20