different rightsholders and authors. Managing opt-outs for so many different interests within 
one book may get overly complicated very fast.  
In any event, creating an opt-out system will need some ways of authenticating whether 
someone has the relevant authority to make choices about inclusion of a work. 
Who would get to use the books data commons? For what? 
A commons might be made publicly available to all, as has been done with datasets like The 
Pile. Another possible design choice is to restrict access only to authorized users and to 
enforce particular responsibilities or obligations in return for authorization. Three particular 
dimensions of permitted uses and users came up in our discussions: 
• Deﬁning and ensuring acceptable and ethical use:  Participants discussed to what 
extent restrictions should be put on use of the resource. In the case of HathiTrust, 
acceptable use is implicitly ensured by limiting access to researchers from member 
institutions; other forms of “gated access” are possible, allowing access only to 
certain types of users and for certain uses.  One can imagine more ﬁne-grained 39
mechanisms, based on a review of the purpose for which datasets are used. This 
imagined resource could become a useful lever to demand responsible development 
and use of AI; alongside “sticks” like legal penalties, this would be a “carrot” that 
could incentivize good behavior. At the same time, drawing the lines around, let alone 
enforcing, “good behavior” would constitute a signiﬁcant challenge.  
• Charging for use to support sustainability of the training corpus itself: While wanting 
to ensure broad access to this resource, it is important to consider economic 
sustainability, including support for continuing to update the resource with new works 
and appropriate tooling for AI training. Requiring some form of payment to use the 
resource could support sustainability, perhaps with different requirements for 
different types of users (e.g., differentiating between non-commercial and 
commercial users, or high-volume, well-resourced users and others).  40
• Ensuring bene ﬁts of AI are broadly shared, including with book authors or 
publishers: The creation of a training resource might lower barriers to the 
development of AI tools, and in that way support broadly shared bene ﬁts by 
facilitating greater competition and mitigating concentration of power. On the other 
hand, just as concentration of technology industries is already a signi ﬁcant challenge, 
AI might not look much different, and the bene ﬁts of this resource may still simply go 
to a few large ﬁrms in “winner takes all-or-most” markets. The workshops discussed 
how, for instance, large commercial users might be expected to contribute to a fund 
that supported contributors of training data, or more generally to fund writers, to 
ensure everyone contributing to the development of AI beneﬁts.  
 For examples of gated access to AI models, see https://huggingface.co/docs/hub/en/models-gated.39
 As an analogy, consider for instance Wikimedia Enterprise, which “build[s] services for high-volume 40
commercial reusers of Wikimedia content” and charges for that access. https://meta.wikimedia.org/
wiki/Wikimedia_Enterprise.
Towards a Books Data Commons for AI Training 18