Dense Process Reward Optimization for Long-Horizon GUI Agents
Provide dense, step-by-step feedback to long-horizon GUI agents by training a dedicated process reward model on step-level critic data.
Source paper
Jiabo Ye, Xi Zhang, Haiyang Xu, Haowei Liu, Junyang Wang, Zhaoqing Zhu, Ziwei Zheng, Feiyu Gao, Junjie Cao, Zhengxi Lu, Jitong Liao, Qi Zheng, Fei Huang, Jingren Zhou, Ming Yan
Generated by AI. The paper’s authors did not write these.
Provide dense, step-by-step feedback to long-horizon GUI agents by training a dedicated process reward model on step-level critic data.
Train robust GUI agents against dynamically generated adversarial disturbances using a curriculum learning framework to systematically expose failure modes.
Enable infinite-context interactions in long-horizon agents by introducing a recurrent visual memory module that passes compressed latent representations between trajectory segments.
This paper presents GUI-Owl and Mobile-Agent-v3, a foundational end-to-end multimodal model and multi-agent framework designed for cross-platform GUI automation (Mobile, PC, Web). The authors introduce a self-evolving data production pipeline that synthesizes large-scale training data through model rollouts, critic-based filtering, and VLM-generated guidance, followed by a scalable reinforcement learning framework using Trajectory-aware Relative Policy Optimization (TRPO) to handle long-horizon tasks.
The core methodology utilizes Qwen2.5-VL as the base model, fine-tuned on a massive dataset generated via a self-evolving pipeline that includes 'Step-Level' and 'Trajectory-Level' critics. The model is further optimized using an asynchronous reinforcement learning framework (Mobile-Agent-v3) that employs TRPO to assign trajectory-level rewards to individual steps, mitigating sparse reward issues in multi-step GUI tasks.