2024-03-14
Excited to share one of the main things I've been working on: scaling towards grounded language agents that can follow natural language instructions across many video game environments—using the same pixels to keyboard/mouse controls as humans do. 1/8 [image]
The Verge
Google DeepMind details SIMA, an AI agent training to learn gaming skills to play like a human; SIMA trained on No Man's Sky, Goat Simulator 3, and other games
using the same pixels to keyboard/mouse controls as humans do. 1/8 [image] @amoufarek : So happy to be able to share what I've been working on with a fantastic team @GoogleDeepMind...
We built our Scalable Instructable Multiworld Agent (SIMA) around the representations of internet-pretrained models, including a video model, and trained them primarily via instruction-conditioned BC, augmented with classifier-free guidance. 4/ [image]
The Verge
Google DeepMind details SIMA, an AI agent training to learn gaming skills to play like a human; SIMA trained on No Man's Sky, Goat Simulator 3, and other games
using the same pixels to keyboard/mouse controls as humans do. 1/8 [image] @amoufarek : So happy to be able to share what I've been working on with a fantastic team @GoogleDeepMind...