Skip to content

Source paper

NitroGen: An Open Foundation Model for Generalist Gaming Agents

Loïc Magne, Anas Awadalla, Guanzhi Wang and 11 more

Loïc Magne, Anas Awadalla, Guanzhi Wang, Yinzhen Xu, Joshua Belofsky, Fengyuan Hu, Joohwan Kim, Ludwig Schmidt, Georgia Gkioxari, Jan Kautz, Yisong Yue, Yejin Choi, Yuke Zhu, Linxi "Jim" Fan

arXiv 2601.02427 · 202625 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper introduces NitroGen, a foundation model for generalist gaming agents trained on a massive dataset of 40,000 hours of gameplay videos with actions extracted from on-screen gamepad overlays. By using a vision-action transformer architecture with flow matching, NitroGen demonstrates strong cross-game generalization and acts as a robust pre-trained backbone for downstream tasks.

How it works (AI summary)

The authors curate an internet-scale dataset by extracting actions from gamepad overlays in videos using template matching and segmentation. The model uses a SigLIP 2 vision encoder and a Diffusion Transformer (DiT) with flow matching to predict chunks of 16 actions conditioned on a single history frame.