Optimizing Agent Performance via Iterative Reinforcement Learning and Data Filtering
Optimize an agent using PPO and heuristic trajectory filtering to bridge the performance gap caused by single-stage supervised fine-tuning.
Source paper
Yuwen Du, Rui Ye, Shuo Tang, Xinyu Zhu, Yijun Lu, Yuzhu Cai, Siheng Chen
Generated by AI. The paper’s authors did not write these.
Optimize an agent using PPO and heuristic trajectory filtering to bridge the performance gap caused by single-stage supervised fine-tuning.
Upgrade the English training data generation pipeline with stricter entity obfuscation and larger subgraph sizes to match the high difficulty of the Chinese set.
Extend a search-only agent framework to support execution environments like Python REPL and file manipulation using synthesized multi-tool reasoning trajectories.
The paper introduces OpenSeeker, the first fully open-source search agent that achieves frontier-level performance comparable to proprietary industrial models. By open-sourcing both the high-quality training data and model weights, it aims to democratize the development of autonomous search agents.
The authors utilize a two-pronged synthesis approach: 'Fact-grounded scalable controllable QA synthesis' to reverse-engineer web graphs into complex multi-hop queries, and 'Denoised trajectory synthesis' using retrospective summarization to provide clean history contexts for the teacher model. The final agent is then trained via Supervised Fine-Tuning (SFT) on raw trajectories to intrinsically learn denoising.