Google DeepMind just introduced an AI agent that plays video games with you

Google DeepMind’s subsidiary, DeepMind, has introduced a groundbreaking AI agent named SIMA. Unlike traditional game AI programmed for victory, SIMA is designed to learn and play video games by following instructions, mimicking human gameplay experiences. Standing for “Scalable, Instructable, Multiworld Agent,” SIMA is focused on broadening its capabilities to play any video game, including open-world scenarios.

In collaboration with eight game developers, including studios like Hello Games and Coffee Stain, DeepMind trained SIMA by plugging it into games like No Man’s Sky and Goat Simulator 3. The process involved creating a custom environment in the Unity engine to teach the AI agent object manipulation through sculpture creation. Additionally, recording human players provided language cues, helping SIMA predict on-screen events based on actions and instructions.

Although currently proficient in over 600 basic skills like turning left and climbing ladders, the ultimate goal is for SIMA to handle more complex tasks within games, such as resource gathering and camp building. While still in the research phase, SIMA’s development marks a significant leap towards creating general, adaptable AI companions that could enhance the gaming experience by offering human-like gameplay interactions.