Generative GUI World Models for Sample-Efficient RL
Improve sample efficiency in reinforcement learning by replacing real-device environments with a generative video world model that simulates graphical user interface transitions.
Time
Compute level
Resources
What the paper used
Prerequisites
Scope
The Gap
The paper relies on a massive 'Device-as-a-Service' (DaaS) infrastructure (Section 2.3.2) to support Online RL, requiring thousands of concurrent connections to real Android/Web environments to generate trajectories.
“In real-world GUI environments, agents are required to navigate dynamic GUI states, stochastic system behaviors, and extended decision-making horizons, where real-time feedback during execution is critical to performance.”
Limitation
The reliance on a large-scale physical/virtual device farm creates a significant resource bottleneck and latency overhead, making the Online RL phase unscalable for researchers without industrial infrastructure.
Proposed Solution
Develop a Generative World Model (e.g., a diffusion or transformer-based video predictor) trained on the Mid-Training corpus to simulate GUI state transitions. Use this differentiable simulator to train the agent via Model-Based Reinforcement Learning (MBRL) before fine-tuning on real devices, drastically reducing the number of real-time interactions needed.
Why This Matters
This would democratize the training of robust GUI agents by removing the dependency on massive device farms, significantly accelerating training throughput and reducing costs.
How to Get Started
How to test it: Hypothesis: A generative GUI world model will improve sample efficiency in online RL. Baseline: Standard online RL using direct interactions with real/emulated device farms. Metric: Success rate on AndroidWorld vs. number of real-world interactions. Stopping condition: If sample efficiency does not improve by at least 30% or if world model hallucination causes catastrophic policy divergence.
- Aggregate existing GUI interaction traces and video recordings into a training dataset for a video prediction model.
- Train a diffusion or transformer-based world model to accurately predict the next graphical user interface state given user actions.
- Train an agent using model-based reinforcement learning within the synthetic simulator before deploying to real devices.
Code linked to this paper on Hugging Face: github.com/inclusionAI/UI-Venus
Suggested by AI. Adapt it to your setup.
Has This Been Done?
Papers like ViMo: A Generative Visual GUI World Model for App Agents and WM-R1: Training GUI Agents to Reason and leverage World Models with Reinforcement Learning directly explore using generative video-based world models for GUI agents. Additionally, Code2World: A GUI World Model via Renderable Code Generation addresses learning state transitions to train agents via model-based techniques.
- ViMo: A Generative Visual GUI World Model for App Agents · 2025 · 44 citations
- WM-R1: Training GUI Agents to Reason and leverage World Models with Reinforcement Learning · 2026 · 1 citations
- Code2World: A GUI World Model via Renderable Code Generation · 2026 · 35 citations
We searched OpenAlex and arXiv for “Improve sample efficiency in reinforcement learning by replacing real-device environments with a generative video world model that simulates graphical user interface transitions.” and “Generative GUI World Models for Sample-Efficient RL” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
UI-Venus-1.5 Technical ReportVeuns-Team, Changlong Gao, Zhangxuan Gu +24 more · 2026
This paper introduces UI-Venus-1.5, an end-to-end multimodal GUI agent designed to navigate mobile and web environments via a unified model. The system is trained through a three-stage pipeline: large-scale Mid-Training on 10B tokens of GUI data, task-specific Offline Reinforcement Learning (RL), and massive Online RL on a Device-as-a-Service infrastructure, culminating in a model merging strategy that unifies specialized capabilities.
Related Ideas
- Interactive Policy Refinement via World Model RLGigaBrain-0: A World Model-Powered Vision-Language-Action Model⛔Very hard
- Reinforcement Learning for Long-Horizon GUI Agent NavigationPenguin-VL: Exploring the Efficiency Limits of VLM with LLM-based Vision Encoders⛔Very hard
- Adversarial Curriculum for Robust GUI ResilienceMobile-Agent-v3: Foundamental Agents for GUI Automation🟠Medium
- Temporal GUI Dynamics Modeling for Animated and Asynchronous InterfacesStep-GUI Technical Report🔴Hard
- Optimizing High-Concurrency Database and Inference Pipelines for Million-Agent SimulationsOASIS: Open Agent Social Interaction Simulations with One Million Agents🟠Medium