Skip to content
AI-generated research idea🟠Medium🔬Significant ResearchRelated work found

Semantic Compressive Memory for Long-Horizon Context

Retain early trajectory evidence for long-horizon agents by compressing aged tool outputs into textual summaries stored in a dedicated memory block.

Time

a couple of months

Compute level

Single GPU or less

Resources

Standard GPU

Prerequisites

transformer architectures

Scope

modify existing code

The Gap

Section 3.3 describes 'Recency-Based Context Retention', where tool outputs outside a window 'K' are simply masked with a null token. While this saves context, it permanently destroys factual evidence gathered early in the trajectory, which is critical for the long-horizon research tasks (up to 600 turns) that MiroThinker aims to solve.

Limitation

The current 'hard masking' context management strategy is a heuristic that risks catastrophic forgetting of early-stage research findings, forcing the agent to re-fetch information if it becomes relevant again later.

Proposed Solution

Replace the binary masking strategy with a 'Compressive Memory' module. When a tool output O_t slides out of the recency window St(K), pass it through a lightweight summarizer (e.g., a small 8B model or the main model itself) to generate a dense textual summary or vector representation. Append this summary to a dedicated 'Long-Term Memory' block in the prompt, allowing the agent to retain high-level insights from early turns without retaining the verbose raw tool output.

Why This Matters

Enables truly effective long-context reasoning where early discoveries influence later decisions, moving beyond simple 'recency bias' to holistic information synthesis.

How to Get Started

  1. Define the sliding window threshold and identify tool outputs that fall outside the recency window.
  2. Integrate a lightweight summarization model to condense aged tool outputs into dense text summaries.
  3. Create a dedicated long-term memory prompt block to store and retrieve these compressed summaries.
  4. Evaluate long-horizon task completion rates on benchmark tasks requiring multi-turn context retention.

Code linked to this paper on Hugging Face: github.com/MiroMindAI/MiroThinker

Suggested by AI. Adapt it to your setup.

Has This Been Done?

Related work found

Trajectory Memory Compression for Long-Horizon Dialogue Agents with Provenance Preservation and Memory in the Age of AI Agents discuss compressing agent trajectories and managing long-horizon agent memory. They do not specifically test the proposed mechanism of replacing sliding window observations with light summarizers into a dedicated prompt block as detailed here.

We searched OpenAlex for “Retain early trajectory evidence for long-horizon agents by compressing aged tool outputs into textual summaries stored in a dedicated memory block.” and “Semantic Compressive Memory for Long-Horizon Context” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling

MiroMind Team, Song Bai, Lidong Bing +51 more · 2025

MiroThinker v1.0 is an open-source research agent designed to push performance boundaries through 'interactive scaling,' where the model is trained to handle deeper and more frequent agent-environment interactions. By utilizing a 256K context window and a three-stage training pipeline involving SFT, DPO, and Reinforcement Learning (GRPO), the agent achieves state-of-the-art results on benchmarks like GAIA and BrowseComp, rivaling proprietary models like GPT-5-high.

Related Ideas

Is this a good idea?

Share onXLinkedIn