Persistent Identity Memory Module
Introduce a global identity anchor via a separate cross-attention block to allow diffusion transformers to maintain long-term subject continuity beyond short-term frame windows.
Time
Compute level
Resources
Prerequisites
Scope
The Gap
Section B.4 explains that the model maintains continuity by conditioning on the last 5 motion frames (short-term memory). It lacks a mechanism to store information beyond this window.
Limitation
Identity drift occurs over long durations; specifically, if a character exits the frame and re-enters later, the model has 'forgotten' their appearance, leading to identity swapping or degradation.
Proposed Solution
Develop a 'Global Identity Anchor' mechanism. Instead of only checking the previous 5 frames, modify the DiT architecture to include a separate cross-attention block that attends to a cached set of feature tokens from the very first frame (or a user-defined reference) at every generation step. This creates a persistent 'long-term memory' channel alongside the autoregressive short-term context.
Why This Matters
Crucial for storytelling and movie production where characters must remain consistent across distinct scenes and over long runtimes, addressing a major failure case in current long-video generation.
How to Get Started
- Modify the Diffusion Transformer architecture to accept cached feature tokens from a designated reference frame.
- Implement an auxiliary cross-attention layer that continuously attends to the global identity anchor throughout generation steps.
- Evaluate identity preservation on benchmark generation datasets containing multi-frame video sequences.
- Compare temporal consistency metrics against the standard short-term autoregressive baseline.
Code linked to this paper on Hugging Face: github.com/vita-epfl/Stable-Video-Infinity
Suggested by AI. Adapt it to your setup.
Has This Been Done?
Papers like Advancing Narrative Long Video Generation via Training-Free Identity-Aware Memory and Gloria: Consistent Character Video Generation via Content Anchors explore mechanisms for maintaining character identity over long video generation using reference features or content anchors. However, they do not specifically propose the exact DiT architecture modification involving a dedicated cross-attention block attending to a cached set of feature tokens from the first frame at every generation step.
- Advancing Narrative Long Video Generation via Training-Free Identity-Aware Memory · 2026 · 8 citations
- Gloria: Consistent Character Video Generation via Content Anchors · 2026 · 4 citations
We searched OpenAlex for “Introduce a global identity anchor via a separate cross-attention block to allow diffusion transformers to maintain long-term subject continuity beyond short-term frame windows.” and “Persistent Identity Memory Module” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
Stable Video Infinity: Infinite-Length Video Generation with Error RecyclingWuyang Li, Wentao Pan, Po-Chien Luan +2 more · 2025
This paper introduces Stable Video Infinity (SVI), a framework designed to generate infinite-length videos with high consistency by addressing the problem of error accumulation in autoregressive models. The authors identify a 'hypothesis gap' where models trained on clean data fail when exposed to their own self-generated errors during inference, and they propose 'Error-Recycling Fine-Tuning' to train the model to actively correct these errors.
Related Ideas
- Sliding-Window Attention with Global Context for Long-Form SynthesisLTX-2: Efficient Joint Audio-Visual Foundation Model🟠Medium
- Dynamic Semantic Attention Sinks for Long-Horizon StreamingLiveTalk: Real-Time Multimodal Interactive Video Diffusion via Improved On-Policy Distillation🟠Medium
- Hierarchical Dynamic Memory Aggregation for Long-Form Video ReasoningV-JEPA-LM: Video-Language Models That Read Dynamics Beyond Static, Text-Aligned Representations🟠Medium
- Entity-Disentangled Memory via SegmentationStoryMem: Multi-shot Long Video Storytelling with Memory🔴Hard
- Text-Guided Multimodal Memory RetrievalStoryMem: Multi-shot Long Video Storytelling with Memory🟠Medium