Quantization-Aware Low-Memory Inference
A post-training quantization technique that applies 4-bit weight compression to Diffusion Transformers to fit large video models into consumer GPU memory.
Source paper
Bing Wu, Chang Zou, Changlin Li, Duojun Huang, Fang Yang, Hao Tan, Jack Peng, Jianbing Wu, Jiangfeng Xiong, Jie Jiang, Linus, Patrol, Peizhen Zhang, Peng Chen, Penghao Zhao, Qi Tian, Songtao Liu, Weijie Kong, Weiyan Wang, Xiao He, Xin Li, Xinchi Deng, Xuefei Zhe, Yang Li, Yanxin Long, Yuanbo Peng, Yue Wu, Yuhong Liu, Zhenyu Wang, Zuozhuo Dai, Bo Peng, Coopers Li, Gu Gong, Guojian Xiao, Jiahe Tian, Jiaxin Lin, Jie Liu, Jihong Zhang, Jiesong Lian, Kaihang Pan, Lei Wang, Lin Niu, Mingtao Chen, Mingyang Chen, Mingzhe Zheng, Miles Yang, Qiangqiang Hu, Qi Yang, Qiuyong Xiao, Runzhou Wu, Ryan Xu, Rui Yuan, Shanshan Sang, Shisheng Huang, Siruis Gong, Shuo Huang, Weiting Guo, Xiang Yuan, Xiaojia Chen, Xiawei Hu, Wenzhi Sun, Xiele Wu, Xianshun Ren, Xiaoyan Yuan, Xiaoyue Mi, Yepeng Zhang, Yifu Sun, Yiting Lu, Yitong Li, You Huang, Yu Tang, Yixuan Li, Yuhang Deng, Yuan Zhou, Zhichao Hu, Zhiguang Liu, Zhihe Yang, Zilin Yang, Zhenzhi Lu, Zixiang Zhou, Zhao Zhong
Generated by AI. The paper’s authors did not write these.
A post-training quantization technique that applies 4-bit weight compression to Diffusion Transformers to fit large video models into consumer GPU memory.
A reward modeling method that penalizes physically impossible motions using trajectory smoothness heuristics during reinforcement learning from human feedback.
Replace simple channel concatenation in video super-resolution networks with 3D temporal cross-attention and a warping-error loss to eliminate shimmering artifacts between frames.
Replace hard top-k block pruning with a learnable soft-gating mechanism using Gumbel-Softmax to prevent abrupt attention drop-offs in complex scenes.
This paper introduces HunyuanVideo 1.5, an open-source 8.3B parameter video generation model that achieves state-of-the-art performance comparable to proprietary models like Sora and Kling. The authors propose a unified Diffusion Transformer (DiT) architecture optimized with a novel Selective and Sliding Tile Attention (SSTA) mechanism and a high-compression 3D VAE, trained via a progressive multi-stage pipeline involving text-to-image, text-to-video, and image-to-video tasks.
The model uses an 8.3B parameter Diffusion Transformer (DiT) with flow matching. Key innovations include: 1) A 3D Causal VAE with 16x spatial and 4x temporal compression; 2) Selective and Sliding Tile Attention (SSTA) to prune redundant tokens and speed up inference; 3) A cascaded Video Super-Resolution (VSR) network to upscale 480p/720p outputs to 1080p; 4) A post-training pipeline incorporating SFT, offline DPO, and online RL with a VLM-based reward model.