Self-Supervised Skill Distillation for Oracle-Free Evolution
Enable self-contained agent evolution by replacing external proprietary oracle teachers with self-distillation and internal self-correction mechanisms.
Source paper
Peng Xia, Jianwen Chen, Hanyang Wang, Jiaqi Liu, Kaide Zeng, Yu Wang, Siwei Han, Yiyang Zhou, Xujiang Zhao, Haifeng Chen, Zeyu Zheng, Cihang Xie, Huaxiu Yao
Generated by AI. The paper’s authors did not write these.
Enable self-contained agent evolution by replacing external proprietary oracle teachers with self-distillation and internal self-correction mechanisms.
Perform dynamic, step-wise skill retrieval indexed by both task descriptions and current environment observations to provide just-in-time reasoning support.
Store and query multimodal embeddings of visual observations in the skill library to enable visual similarity-based skill retrieval.
The paper introduces SKILLRL, a framework that enables LLM agents to learn from past experiences by distilling successful and failed trajectories into a hierarchical skill library (General and Task-Specific) using a teacher model. This library is then used to augment the agent's context during Reinforcement Learning (specifically GRPO), where the skill library recursively evolves based on validation failure modes to continuously improve performance.
The method utilizes Experience-based Skill Distillation to convert raw trajectories into text-based skills using a powerful teacher model (OpenAI o3). It employs a cold-start Supervised Fine-Tuning (SFT) phase to teach the agent to use these skills, followed by an online RL phase using Group Relative Policy Optimization (GRPO) where the skill library is dynamically updated via recursive evolution.