Skip to content
AI-generated research idea🔴Hard🔬Significant ResearchRelated work found

Infinite-Horizon Attention via Compressed Memory Caching in Hierarchical Denoising

Integrate continuous memory compression into hierarchical video denoising to maintain an infinite-horizon context alongside a dense recent sliding window.

Time

4-5 days

Compute level

Multiple GPUs

Resources

8x NVIDIA A100 GPUs to retrain and evaluate the memory compression module within the hierarchical denoising pipeline.

What the paper used

Fine-tuning Wan2.1-1.3B with a Wan2.1-14B teacher critic model for 20k steps with total batch size 64 on 5-second video clips.

Prerequisites

Transformer architectures, KV cache management, state space models or memory augmented networks.

Scope

New model architecture component and substantial retraining.

The Gap

In the Implementation Details (Section 4.1), the autoregressive generation relies on a sliding-window KV cache with a strict, constant attention window of 5 seconds.

“At inference time, we employ a sliding-window KV cache with a constant attention window of 5 s.”
From the paper, Section 4.1, page 7. Matched against the PDF text.

Limitation

Truncating the context to exactly 5 seconds means the model suffers from complete amnesia regarding earlier events. This prevents the generation of long-term narrative dependencies, returning characters, or cyclic actions spanning longer than the 5-second window.

Proposed Solution

Integrate a continuous memory compression module (such as streaming summarization tokens or a localized Token Turing Machine) into the hierarchical denoising pipeline. This allows the KV cache to store a dense 5-second window alongside a compressed, infinite-horizon representation of all prior blocks.

Why This Matters

This would transition the model from simply generating 'infinite continuous video' to generating 'narratively coherent infinite video', which is a critical step toward true interactive world models.

How to Get Started

How to test it: Hypothesize that incorporating compressed long-horizon memory tokens into the KV cache reduces character identity drift over 30s+ generation. Compare against HiAR with standard 5s sliding window using long-video VBench semantic consistency and drift metrics. Stop if character consistency does not improve or inference throughput drops by >20%.

  1. Examine the baseline sliding-window KV cache implementation and memory usage profiles in video foundation models.
  2. Implement a continuous memory compression module using streaming summarization tokens or localized memory banks.
  3. Integrate the compressed memory caching mechanism into the hierarchical denoising autoregressive pipeline.
  4. Test long-horizon generation stability and memory footprint reduction against standard fixed-window baselines.

Code linked to this paper on Hugging Face: github.com/Jacky-hate/HiAR

Suggested by AI. Adapt it to your setup.

Has This Been Done?

Related work found

Works like MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokens and Addressable Memory for Video World Models explore memory caching and long-horizon video generation mechanisms. However, they do not specifically integrate a continuous memory compression module combining a dense short-term window with a compressed infinite-horizon representation in hierarchical denoising as proposed.

We searched OpenAlex and arXiv for “Integrate continuous memory compression into hierarchical video denoising to maintain an infinite-horizon context alongside a dense recent sliding window.” and “Infinite-Horizon Attention via Compressed Memory Caching in Hierarchical Denoising” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

HiAR: Efficient Autoregressive Long Video Generation via Hierarchical Denoising

Kai Zou, Dian Zheng, Hongbo Liu +3 more · 2026

This paper proposes HiAR, an autoregressive framework for long video generation that mitigates distribution drift by utilizing hierarchical denoising. By conditioning generated blocks on context at matched noise levels instead of fully clean frames, it effectively reduces error propagation while enabling pipelined parallel inference.

Related Ideas

Is this a good idea?

Share onXLinkedIn