Generative GUI World Models for Sample-Efficient RL
Improve sample efficiency in reinforcement learning by replacing real-device environments with a generative video world model that simulates graphical user interface transitions.
Source paper
Veuns-Team, Changlong Gao, Zhangxuan Gu, Yulin Liu, Xinyu Qiu, Shuheng Shen, Yue Wen, Tianyu Xia, Zhenyu Xu, Zhengwen Zeng, Beitong Zhou, Xingran Zhou, Weizhi Chen, Sunhao Dai, Jingya Dou, Yichen Gong, Yuan Guo, Zhenlin Guo, Feng Li, Qian Li, Jinzhen Lin, Yuqi Zhou, Linchao Zhu, Liang Chen, Zhenyu Guo, Changhua Meng, Weiqiang Wang
Generated by AI. The paper’s authors did not write these.
Improve sample efficiency in reinforcement learning by replacing real-device environments with a generative video world model that simulates graphical user interface transitions.
Preserve specialist capabilities in unified models without destructive merging conflicts by routing tasks through a mixture of modular adapters.
Align offline reinforcement learning with long-term task success by optimizing trajectories using direct preference optimization over complete sequences.
This paper introduces UI-Venus-1.5, an end-to-end multimodal GUI agent designed to navigate mobile and web environments via a unified model. The system is trained through a three-stage pipeline: large-scale Mid-Training on 10B tokens of GUI data, task-specific Offline Reinforcement Learning (RL), and massive Online RL on a Device-as-a-Service infrastructure, culminating in a model merging strategy that unifies specialized capabilities.
The model utilizes a Vision-Language Model (Qwen3-VL) backbone. The training pipeline progresses from supervised pre-training on a massive GUI corpus to Offline RL using step-wise rewards, followed by Online RL using Group Relative Policy Optimization (GRPO) on real devices to optimize full trajectories. Finally, domain-specific checkpoints (Web, Mobile, Grounding) are consolidated using TIES-Merge.