Temporal-Consistency Enhanced Super-Resolution
Replace simple channel concatenation in video super-resolution networks with 3D temporal cross-attention and a warping-error loss to eliminate shimmering artifacts between frames.
Time
Compute level
Resources
Prerequisites
Scope
The Gap
Section 3.2 describes the Video Super-Resolution (VSR) network, which takes 'low-resolution latents and noise' as input via channel concatenation. While it aligns spatially, the paper does not mention explicit mechanisms to handle temporal consistency during upscaling. Upscalers often introduce 'shimmering' artifacts where high-frequency textures (like grass or hair) change randomly between frames.
Limitation
The current VSR approach treats frames semi-independently in the latent space, potentially introducing high-frequency temporal flickering in fine details.
Proposed Solution
Replace the simple channel concatenation in the VSR network with a 3D Temporal Cross-Attention mechanism. Inject the Low-Resolution (LR) latent features as keys/values into the VSR transformer blocks. Furthermore, introduce a temporal consistency loss (e.g., warping error based on optical flow) specifically during the training of the VSR module to penalize pixel-shimmering.
Why This Matters
Eliminating flickering artifacts is essential for video generation to be perceived as 'high definition' rather than just 'high resolution.'
How to Get Started
- Set up the baseline video super-resolution codebase and verify inference pipelines using synthetic datasets.
- Implement the 3D Temporal Cross-Attention module by modifying the transformer block architecture.
- Integrate an optical flow estimation model to compute warping error for the temporal consistency loss.
- Train the modified model on a large-scale video dataset and evaluate against baseline metrics using PSNR, SSIM, and temporal consistency scores.
Code linked to this paper on Hugging Face: github.com/Tencent-Hunyuan/HunyuanVideo-1.5
Suggested by AI. Adapt it to your setup.
Has This Been Done?
Papers like Enhancing temporal consistency in video super-resolution using optical flow-guided deep networks and Temporal Consistency Learning of Inter-Frames for Video Super-Resolution explore optical flow-guided losses and temporal consistency in video super-resolution. However, they do not specifically evaluate the combination of 3D temporal cross-attention for LR feature injection with warping error penalties.
- Enhancing temporal consistency in video super-resolution using optical flow-guided deep networks · 2026 · 0 citations
- Temporal Consistency Learning of Inter-Frames for Video Super-Resolution · 2022 · 58 citations
We searched OpenAlex for “Replace simple channel concatenation in video super-resolution networks with 3D temporal cross-attention and a warping-error loss to eliminate shimmering artifacts between frames.” and “Temporal-Consistency Enhanced Super-Resolution” and an AI compared the idea with the top 6 results. A small search can miss relevant work.
Source Paper
HunyuanVideo 1.5 Technical ReportBing Wu, Chang Zou, Changlin Li +78 more · 2025
This paper introduces HunyuanVideo 1.5, an open-source 8.3B parameter video generation model that achieves state-of-the-art performance comparable to proprietary models like Sora and Kling. The authors propose a unified Diffusion Transformer (DiT) architecture optimized with a novel Selective and Sliding Tile Attention (SSTA) mechanism and a high-compression 3D VAE, trained via a progressive multi-stage pipeline involving text-to-image, text-to-video, and image-to-video tasks.
Related Ideas
- Temporal-Consistent Quantization for Video StabilityTurboDiffusion: Accelerating Video Diffusion Models by 100-200 Times🟠Medium
- Efficient Mobile Latent UpscalingMobile-O: Unified Multimodal Understanding and Generation on Mobile Device🟡Fairly easy
- Spatiotemporal Prism: Unified Video AutoencodingThe Prism Hypothesis: Harmonizing Semantic and Pixel Representations via Unified Autoencoding🔴Hard
- Entropy-Aware Adaptive 3D-ResamplerMiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe🟠Medium
- Temporal Redundancy Exploitation via Feature-Space PropagationPersonaLive! Expressive Portrait Image Animation for Live Streaming🔴Hard