Skip to content

Source paper

BitDance: Scaling Autoregressive Generative Models with Binary Tokens

Yuang Ai, Jiaming Han, Shaobin Zhuang and 7 more

Yuang Ai, Jiaming Han, Shaobin Zhuang, Weijia Mao, Xuefeng Hu, Ziyan Yang, Zhenheng Yang, Huaibo Huang, Xiangyu Yue, Hao Chen

arXiv 2602.14041 · 202619 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper introduces BitDance, a scalable autoregressive (AR) image generation framework that replaces standard vector quantization codebook indices with high-entropy binary tokens (up to 2^256 states). To handle this massive vocabulary efficiently, the authors propose a "Binary Diffusion Head" that models the tokens via continuous diffusion and a "Next-Patch Diffusion" strategy to predict multiple tokens in parallel, achieving state-of-the-art performance with significant speedups over existing AR and diffusion models.

How it works (AI summary)

The model uses a pure CNN-based binary tokenizer with Lookup-Free Quantization (LFQ) to create high-entropy discrete representations. The backbone is an AR Transformer that predicts patches of these tokens. Instead of a softmax classifier, the output head is a lightweight Diffusion Transformer (DiT) that models the joint distribution of the binary bits in continuous space using Rectified Flow, followed by hard binarization.