Google DeepMind details SIMA, an AI agent training to learn gaming skills to play like a human; SIMA trained on No Man's Sky, Goat Simulator 3, and other games
using the same pixels to keyboard/mouse controls as humans do. 1/8 [image] @amoufarek : So happy to be able to share what I've been working on with a fantastic team @GoogleDeepMind Introducing SIMA! 🤖🎮🥳🚀 Working on AI R&D as a gamer and former game developer, the SIMA project is my happy place 😀 I'll be at #GDC2024 next week, reach out if you'd like to chat! Andrew Lampinen / @andrewlampinen : We built our Scalable Instructable Multiworld Agent (SIMA) around the representations of internet-pretrained models, including a video model, and trained them primarily via instruction-conditioned BC, augmented with classifier-free guidance. 4/ [image] Shane Legg / @shanelegg : Our research project SIMA is creating a general, natural language instructable, multi 3D game-playing AI agent. The agent can carry out a wide range of tasks in virtual worlds, making AI more adaptable, helpful & fun! https://dpmd.ai/sima-1 [video] Edward Grefenstette / @egrefen : I've really enjoyed seeing this project produce an agent which does more and more amazing and diverse things every week. A great reminder that games remain an excellent medium to study generalization and complex behavior. Great job, SIMA team! @googledeepmind : SIMA needs only the images provided by the 3D environment and natural-language instructions given by the user. 🖱️ With mouse and keyboard outputs, it is evaluated across 600 skills, spanning areas like navigation and object interaction - such as “turn left” or “chop down tree.”... [video] @googledeepmind : We partnered with gaming studios to train SIMA (Scalable Instructable Multiworld Agent) on @NoMansSky, @Teardowngame, @ValheimGame and others. 🎮 These offer a wide range of distinct skills for it to learn, from flying a spaceship to crafting a helmet. https://deepmind.google/... LinkedIn: Emil Protalinski : Google DeepMind has unveiled an AI agent that learns how you play video games to help you, not beat you (for now). … Ali Arsanjani, PhD : 🚀 super excited to announce Google DeepMind #SIMA : Scalable Instructable Multiworld Agent. This generalist AI agent is focused on 3D virtual settings. … Forums: r/Futurology : Google DeepMind : Introducing SIMA, a Scalable Instructable Multiworld Agent r/Games : Introducing SIMA, a Scalable Instructable Multiworld Agent