Skip to content

Source paper

Cheers: Decoupling Patch Details from Semantic Representations Enables Unified Multimodal Comprehension and Generation

Yichen Zhang, Da Peng, Zonghao Guo and 19 more

Yichen Zhang, Da Peng, Zonghao Guo, Zijian Zhang, Xuesong Yang, Tong Sun, Shichu Sun, Yidan Zhang, Yanghao Li, Haiyan Zhao, Wang Xu, Qi Shi, Yangang Sun, Chi Chen, Shuo Wang, Yukun Yan, Xu Han, Qiang Ma, Wei Ke, Liang Wang, Zhiyuan Liu, Maosong Sun

arXiv 2603.12793 · 202628 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

The paper introduces CHEERS, a unified multimodal model that seamlessly integrates both visual comprehension and high-fidelity image generation within a single transformer framework. Its primary contribution is a novel architecture that decouples patch-level high-frequency visual details from low-frequency semantic representations, stabilizing multimodal reasoning while enhancing image synthesis through adaptive detail injection.

How it works (AI summary)

The model employs a unified vision tokenizer combining a VAE and SigLIP2-ViT with 2D token compression to extract semantic features. These features are processed by a 1.5B parameter LLM backbone that uses autoregressive decoding for text and a cascaded flow-matching head for continuous-time image generation, dynamically injecting high-frequency detail residuals during the synthesis process.