Skip to content

Source paper

VibeVoice Technical Report

Zhiliang Peng, Jianwei Yu, Wenhui Wang and 10 more

Zhiliang Peng, Jianwei Yu, Wenhui Wang, Yaoyao Chang, Yutao Sun, Li Dong, Yi Zhu, Weijiang Xu, Hangbo Bao, Zehua Wang, Shaohan Huang, Yan Xia, Furu Wei

arXiv 2508.19205 · 2025134 upvotes on Hugging Face

Research ideas from this paper (4)

Generated by AI. The paper’s authors did not write these.

AI summary

VIBEVOICE is a novel speech synthesis framework designed for generating long-form, multi-speaker conversational audio, supporting sequences up to 90 minutes. It introduces an ultra-high 3200x compression causal speech tokenizer that enables efficient processing of long sequences within a Large Language Model (LLM) framework using next-token diffusion.

How it works (AI summary)

The architecture integrates separate acoustic and semantic tokenizers with a pre-trained LLM (Qwen2.5) that processes interleaved voice font features and text scripts. A lightweight, four-layer token-level diffusion head is then conditioned on the LLM's hidden states to iteratively predict and refine continuous Variational Autoencoder (VAE) features for high-fidelity audio reconstruction.