Introducing Ego-Exo4D: A foundational dataset for research on video learning and multimodal perception

The Ego-Exo4D consortium, consisting of FAIR and university partners, has conducted a comprehensive study involving over 800 skilled participants across various countries to capture perspectives on human activities. The consortium is set to open source the collected data and annotations for use in novel benchmark tasks, with a public benchmark challenge planned for the next year. The datasets will be vital for advancing AI understanding of human skills in video and have implications for future technologies such as augmented reality systems, robot learning, and social networks. The release of this data aims to provide tools for the broader research community to explore ego-exo video, multimodal activity recognition, and beyond, addressing the limitations of existing datasets and learning paradigms in this area.