One of the world’s largest AI training datasets is about to get bigger and ‘substantially better’

EleutherAI, a key player in the development of large language model (LLM) training datasets, has faced legal and ethical scrutiny due to copyright and data licensing concerns, putting a spotlight on the significant impact of these datasets on popular language models like GPT-4 and Llama.

Despite legal challenges, EleutherAI is collaborating with organizations such as the University of Toronto and the Allen Institute for AI to develop an updated version of the Pile dataset, expected to be larger, better, and more diverse than the original.

The debate around AI training data has intensified following the release of ChatGPT and has raised complex ethical and legal issues, including concerns about copyright, impact on creative workers, and the need for greater visibility and documentation of training datasets for safer and ethical use of AI models.