Neural Explicit Memory for Consistent Multi-Agent World Simulation
Augment diffusion transformers with an external spatial memory module to persist environment states and prevent hallucination during long-horizon generation.
Source paper
Georgy Savva, Oscar Michel, Daohan Lu, Suppakit Waiwitlikhit, Timothy Meehan, Dhairya Mishra, Srivats Poddar, Jack Lu, Saining Xie
Generated by AI. The paper’s authors did not write these.
Augment diffusion transformers with an external spatial memory module to persist environment states and prevent hallucination during long-horizon generation.
Replace global player attention with a sparse or graph-based mechanism to scale multi-agent world simulations to ten or more players efficiently.
Fine-tune synthetic multi-player video generators on real human gameplay using low-rank adaptation to bridge the distribution gap between bots and humans.
The paper introduces Solaris, a multiplayer video world model for Minecraft capable of simulating consistent multi-view observations for two agents. To achieve this, the authors built a scalable data generation engine (SolarisEngine), collected a large synthetic dataset, and trained a modified Diffusion Transformer (DiT) using a novel "Checkpointed Self Forcing" technique to enable memory-efficient long-horizon autoregressive training.
The model is a Video Diffusion Transformer (DiT) based on Matrix Game 2.0, modified to process joint states of multiple players via a shared self-attention mechanism. Training follows a staged pipeline: single-player pretraining, bidirectional multiplayer finetuning, causal masking, and finally Checkpointed Self Forcing, which allows gradients to flow through long autoregressive chains by recomputing intermediate states to save memory.