EleutherAI is an decrentalized artificial intelligence research collective founded in July 2020. EleutherAI made headlines in 2021 after its GPT-3 replication project GPT-Neo produced the most powerful autoregressive language model freely available online. History While EleutherAI initially turned down funding offers, preferring to use Google's TPU Research Cloud Program to source their compute, by early 2021 they had accepted funding from CoreWeave (a small cloud computing company) and SpellML (a cloud infrastructure company) in the form of access to powerful GPU clusters that are necessary for large scale machine learning research. On June 9, 2021, Eleuther released GPT-J-6B, a artificial intelligence language model built 6 billion parameter model and based on The Pile. On Feb 10, 2022, they released GPT-NeoX-20B, a similar model, based on a 20 billion parameter data set, again based on the Pile. Research and technologies According to their website, EleutherAI is a "decentralized grassroots collective of volunteer researchers, engineers, and developers focused on AI alignment, scaling, and open source AI research". While they do not sell any of their technologies as products, they publish the results of their research in academic venues, write blog posts detailing their ideas and methodologies, and provide trained models for anyone to use for free. EleutherAI is a "decentralized grassroots collective of volunteer researchers, engineers, and developers focusing on AI alignment, scaling, and open source AI research," according to its website. Although they don't market any of their technologies as products, they do publish the findings of their research in academic journals, discuss their concepts and methods in blog postings, and offer trained models for use by anybody at no cost. The Pile The Pile is an open-source 825 gigibyte language modelling data set that is split into 22 smaller datasets. References Language modeling Artificial intelligence laboratories Deep learning Applied machine learning