Skip to content

Source paper

PyVision-RL: Forging Open Agentic Vision Models via RL

Shitian Zhao, Shaoheng Lin, Ming Li and 4 more

Shitian Zhao, Shaoheng Lin, Ming Li, Haoquan Zhang, Wenshuo Peng, Kaipeng Zhang, Chen Wei

arXiv 2602.20739 · 202624 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper introduces PyVision-RL, a framework for training open-weight multimodal agents (based on Qwen2.5-VL) to effectively use Python tools for image and video understanding. To address 'interaction collapse'—where models stop using tools after RL fine-tuning—the authors propose an oversampling-filtering-ranking training strategy and an accumulative tool reward mechanism.

How it works (AI summary)

The model uses Group Relative Policy Optimization (GRPO) with Python as a primitive tool. Key techniques include 'On-Demand Context Construction' for videos (keeping video in runtime and sampling frames via code) and a modified reward function that incentivizes multi-turn interactions. Training data is selected via a curriculum based on reward variance.