Dynamic Step-Wise Skill Retrieval
Perform dynamic, step-wise skill retrieval indexed by both task descriptions and current environment observations to provide just-in-time reasoning support.
Time
Compute level
Resources
What the paper used
Prerequisites
Scope
The Gap
Equation (4) in Section 3.2 defines skill retrieval based solely on the task description embedding (ed): Sret = TopK(sim(ed, es)). This means the set of retrieved skills is static for the entire episode.
“At inference, given a task description d, the agent retrieves relevant skills to augment its context.”
Limitation
In multi-step tasks (e.g., ALFWorld), the skills required change dynamically (e.g., 'Navigation' skills are needed first, 'Manipulation' skills later). Static retrieval fills the context window with skills that may be relevant to the *task* but irrelevant to the *current state*, wasting tokens and potentially confusing the agent.
Proposed Solution
Modify the inference loop to perform dynamic retrieval at regular intervals or upon sub-goal completion. The retrieval query should be a function of both the task description and the current observation/history (ed oplus eot), ensuring that the agent has access to 'Just-In-Time' skills relevant to the immediate bottleneck.
Why This Matters
This would significantly improve token efficiency and reasoning accuracy in long-horizon tasks by reducing context noise and providing precise, state-relevant guidance.
How to Get Started
How to test it: We hypothesize that dynamic step-wise skill retrieval improves multi-step execution over static task-level retrieval. The baseline is static retrieval, measured by trajectory token efficiency and ALFWorld success rate, with a 5% improvement required to continue.
- Modify the inference loop to query the SkillBank at regular time intervals and sub-goal boundaries.
- Construct a joint retrieval query embedding combining task description vectors and current observation vectors.
- Index existing skill libraries using vector databases like FAISS with multi-modal embedding support.
- Evaluate task success rates on long-horizon benchmarks like Mind2Web comparing static versus dynamic retrieval.
Code linked to this paper on Hugging Face: github.com/aiming-lab/SkillRL
Suggested by AI. Adapt it to your setup.
Has This Been Done?
The paper 'Just-in-Time Skill Retrieval via Long-Horizon State Tracking' directly proposes dynamic skill retrieval based on state tracking and task descriptions. Other works like 'VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning' explore iterative retrieval, but the first result essentially executes the proposed dynamic step-wise skill retrieval framework.
- Just-in-Time Skill Retrieval via Long-Horizon State Tracking · 2026 · 0 citations
- VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning · 2026 · 6 citations
We searched OpenAlex for “Perform dynamic, step-wise skill retrieval indexed by both task descriptions and current environment observations to provide just-in-time reasoning support.” and “Dynamic Step-Wise Skill Retrieval” and an AI compared the idea with the top 6 results. A small search can miss relevant work.
Source Paper
SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement LearningPeng Xia, Jianwen Chen, Hanyang Wang +10 more · 2026
The paper introduces SKILLRL, a framework that enables LLM agents to learn from past experiences by distilling successful and failed trajectories into a hierarchical skill library (General and Task-Specific) using a teacher model. This library is then used to augment the agent's context during Reinforcement Learning (specifically GRPO), where the skill library recursively evolves based on validation failure modes to continuously improve performance.
Related Ideas
- Dynamic Multi-Step Retrieval for Long-Horizon TasksRemember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution🟠Medium
- Distilling Explicit Reasoning into Latent SpaceRobust-R1: Degradation-Aware Reasoning for Robust Visual Understanding🟡Fairly easy
- Hierarchical Retrieval-Augmented Self-Improvement Memory for Extended Optimization HorizonsSelf-Improvement Memory Enables Continual Recursive Self-Improvement🟡Fairly easy
- Dynamic Working-Memory Pruning for Search Agents to Overcome Reinforcement Learning SaturationK2-Horizon: A Frontier Fleet of Open Source Language Models🟡Fairly easy
- Compressive Memory for Agentic ContextsGLM-5: from Vibe Coding to Agentic Engineering🟠Medium