Skip to content

Source paper

MOSS-Audio-Tokenizer: Scaling Audio Tokenizers for Future Audio Foundation Models

Yitian Gong, Kuangwei Chen, Zhaoye Fei and 9 more

Yitian Gong, Kuangwei Chen, Zhaoye Fei, Xiaogui Yang, Ke Chen, Yang Wang, Kexin Huang, Mingshu Chen, Ruixiao Li, Qingyuan Cheng, Shimin Li, Xipeng Qiu

arXiv 2602.10934 · 202642 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper introduces MOSS-Audio-Tokenizer, a unified discrete audio tokenizer designed for audio foundation models. It utilizes a homogeneous, purely Transformer-based architecture (CAT) without CNNs, trained end-to-end on 3 million hours of audio to achieve state-of-the-art reconstruction and generation capabilities.

How it works (AI summary)

The model (CAT) replaces traditional CNN-based encoders with a causal Transformer architecture using sliding-window attention and patchifying operations. It employs Residual Vector Quantization (RVQ) and is optimized jointly using reconstruction loss, adversarial discriminators, and a semantic alignment loss via an auxiliary LLM.