On making an open-source pre-trained base model– I think the critical first step is good training data. Much of humanity's knowledge is NOT on the internet, but in unscanned libraries– so I wanna get those

What training data do you think should go into the next great base model?