Skip to content

Source paper

IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech System

Wei Deng, Siyi Zhou, Jingchen Shu and 2 more

Wei Deng, Siyi Zhou, Jingchen Shu, Jinchao Wang, Lu Wang

arXiv 2502.05512 · 20256 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper presents IndexTTS, an industrial-level zero-shot Text-to-Speech system designed for high efficiency, controllability, and stability, particularly in Chinese and English scenarios. The authors introduce a hybrid character-pinyin modeling approach to handle polyphonic ambiguity and replace standard Vector Quantization with Finite-Scalar Quantization (FSQ) to ensure high codebook utilization.

How it works (AI summary)

The model utilizes a decoder-only Transformer (SpeechLLM) conditioned on a Conformer-encoded speech prompt to generate acoustic tokens from text. It employs a hybrid text tokenizer (mixing BPE and pinyin), FSQ for acoustic tokenization to prevent codebook collapse, and a BigVGAN2 decoder to reconstruct high-quality waveforms from the latent representations.