Skip to content

Source paper

Fish Audio S2 Technical Report

Shijia Liao, Yuxuan Wang, Songting Liu and 11 more

Shijia Liao, Yuxuan Wang, Songting Liu, Yifan Cheng, Ruoyi Zhang, Tianyu Li, Shidong Li, Yisheng Zheng, Xingwei Liu, Qingzheng Wang, Zhizhuo Zhou, Jiahua Liu, Xin Chen, Dawei Han

arXiv 2603.08823 · 202620 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper introduces Fish Audio S2, an open-source text-to-speech (TTS) system that achieves multi-speaker, multi-turn generation and fine-grained natural language control over vocal features (e.g., emotion, prosody). The core contribution is an end-to-end framework that seamlessly integrates large-scale data curation, a highly optimized inference engine, and a novel reinforcement learning alignment process that reuses data filtering models as reward signals.

How it works (AI summary)

The system employs a Dual-Autoregressive (Dual-AR) architecture: a 'Slow AR' (Qwen3-4B) generates semantic speech tokens, while a lightweight 'Fast AR' generates fine-grained acoustic codebook tokens. To optimize naturalness and instruction-following, the model undergoes reinforcement learning using a GRPO-variant, utilizing the exact same speech quality and ASR models from the data curation pipeline as multi-dimensional reward signals to avoid distribution shifts.