text:  There are certain technical challenges in using books in AI training as well, given the nature of the 
  8 
  format. First, one must address whether a book is already in digital form. For the vast majority of books, 
  that is not the case. One ﬁrst needs to digitize the book, and convert it to a digital text ﬁle using optical 
  character recognition (OCR), or use a born-digital version (although we return to speciﬁc limitations on 
  that approach below). Second, once a book is in digital text form, it must be converted into a text format 
  that is suitable for AI training. Text conversion tools transfer the content of books into complete text 
  ﬁles, which is akin to the type of conversion that must be done between a Microsoft Word or Adobe PDF 
  ﬁle format and a simple .txt format. This conversion is generally not adequate for the purpose of AI 
  training; researchers have found that post-processing is required to ensure these text ﬁles are properly 
  formatted for the purposes of tokenization. For example, when building the dataset known as The Pile, 
  researchers had to modify an existing epub-to-text converter tool to ensure that document structure 
  across chapters was preserved to match the table of contents, that tables of data were correctly 
  rendered, to convert numbered lists from digitally legible lists of “1\.” to “1.”, and to replace unicode 
  punctuation with ascii punctuation. See Discussion in 4.3.2 in Bandy, Jack, and Nicholas Vincent. 
  *Addressing “Documentation Debt” in Machine Learning Research: A Retrospective Datasheet for*
  *BookCorpus*. 2021, https://arxiv.org/pdf/2105.05241.pdf. and C.16 of The Pile documentation in Gao, 
  Leo, et al.*The Pile: An 800GB Dataset of Diverse Text for Language Modeling*, https://arxiv.org/pdf/ 
  2101.00027.pdf. 