Importance-Weighted Semantic Memory Compaction for Ultra-Long Horizon Tasks
Implement an adaptive, importance-scored memory hierarchy using semantic retrieval and pinned global states to prevent critical loss in ultra-long horizon tasks.
Time
Compute level
Resources
What the paper used
Prerequisites
Scope
The Gap
In Section 3.2 and Section 5.1, Gamer manages long-term context by updating a memory summary buffer capped at 512 tokens. When the token budget is reached, the system enforces a strict first-in, first-out (FIFO) eviction rule where the oldest complete summaries are permanently removed from the conditioning prompt.
“When the memory budget is exceeded, the oldest complete summaries are removed in temporal order.”
Limitation
FIFO eviction causes critical early-episode context (such as overarching quest objectives, persistent item states, or structural environmental constraints) to be permanently discarded during multi-stage quests exceeding 30-100 minutes.
Proposed Solution
Replace the FIFO queue with an adaptive, importance-scored memory hierarchy. Implement a two-tiered memory manager: (1) a pinned global-state register retaining immutable quest goals and key inventory states, and (2) a semantic retrieval module that computes cross-attention relevance between candidate historical summaries and the current task state, evicting low-relevance intermediate summaries rather than strictly older ones.
Why This Matters
Prevents catastrophic memory loss over extended multi-objective workflows, allowing agents to solve complex questlines involving frequent backtracking and delayed prerequisite resolution.
How to Get Started
How to test it: Test whether replacing FIFO eviction (512-token budget) with a pinned global-state register and cross-attention semantic retrieval improves ultra-long-horizon state retention by comparing against the FIFO Gamer-RL baseline on Opportunity Makes the Thief (Part I) (30 min), Bestial Ferocity (100 min), and the 50-hour main-storyline evaluation, measuring milestone accuracy and autonomous completion rate. Stop if overall accuracy on Bestial Ferocity does not surpass Gamer-RL's 83.6% or if semantic retrieval pushes Streaming-vLLM request latency above the 200 ms step budget.
- Collect offline trajectory datasets featuring long-horizon interaction tasks and episodic memory constraints.
- Design the pinned global-state register for storing immutable quest goals and key inventory states.
- Implement a cross-attention relevance scoring module to evaluate historical summaries against current task states.
- Compare the semantic retrieval memory manager against standard FIFO queue eviction baselines on long-horizon success metrics.
Code linked to this paper on Hugging Face: github.com/RainBowLuoCS/Gamer
Suggested by AI. Adapt it to your setup.
Has This Been Done?
Imprint: Online Memory Compression for Long-Horizon Egocentric QA and Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management explore advanced memory compression and management for long-horizon tasks, but do not implement the specific two-tiered pinned global-state register combined with cross-attention relevance retrieval proposed here.
- Imprint: Online Memory Compression for Long-Horizon Egocentric QA · 2026 · 1 citations
- Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management · 2026 · 12 citations
We searched OpenAlex and arXiv for “Implement an adaptive, importance-scored memory hierarchy using semantic retrieval and pinned global states to prevent critical loss in ultra-long horizon tasks.” and “Importance-Weighted Semantic Memory Compaction for Ultra-Long Horizon Tasks” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game AgentRun Luo, Kuan Zhang, Qi Deng +24 more · 2026
This paper presents Gamer, a generalist Vision-Language-Action (VLA) foundation model designed for real-time, long-horizon interactive 3D and 2D gaming environments using raw screen observations and standard keyboard-mouse actions. The system employs a three-stage training pipeline comprising continued pre-training (CPT) with trajectory debiasing, multi-level supervised fine-tuning (SFT) with sparse reasoning and memory compaction, and Temporal-Aware Group Relative Policy Optimization (TA-GRPO) using game-native verifiers. To achieve sub-100ms latency necessary for real-time closed-loop control and rollout generation, the authors develop Streaming-vLLM, featuring stateful KV-cache reuse, incremental RoPE cache updates, and FSM-guided speculative decoding.
Related Ideas
- Dynamic Multi-Step Retrieval for Long-Horizon TasksRemember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution🟠Medium
- Adaptive Attention-Based Memory CompressionHelios: Real Real-Time Long Video Generation Model🟠Medium
- Hierarchical Retrieval-Augmented Self-Improvement Memory for Extended Optimization HorizonsSelf-Improvement Memory Enables Continual Recursive Self-Improvement🟡Fairly easy
- Dynamic Working-Memory Pruning for Search Agents to Overcome Reinforcement Learning SaturationK2-Horizon: A Frontier Fleet of Open Source Language Models🟡Fairly easy
- Semantic Compressive Memory for Long-Horizon ContextMiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling🟠Medium