Skip to content

Source paper

LTX-2: Efficient Joint Audio-Visual Foundation Model

Yoav HaCohen, Benny Brazowski, Nisan Chiprut and 26 more

Yoav HaCohen, Benny Brazowski, Nisan Chiprut, Yaki Bitterman, Andrew Kvochko, Avishai Berkowitz, Daniel Shalem, Daphna Lifschitz, Dudu Moshe, Eitan Porat, Eitan Richardson, Guy Shiran, Itay Chachy, Jonathan Chetboun, Michael Finkelson, Michael Kupchick, Nir Zabari, Nitzan Guetta, Noa Kotler, Ofir Bibi, Ori Gordon, Poriya Panet, Roi Benita, Shahar Armon, Victor Kulikov, Yaron Inger, Yonatan Shiftan, Zeev Melumian, Zeev Farbman

arXiv 2601.03233 · 202650 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper presents LTX-2, an efficient, open-source multimodal foundation model capable of jointly generating synchronized video and audio from text prompts. By utilizing an asymmetric dual-stream transformer architecture with bidirectional cross-attention, the model achieves state-of-the-art audiovisual alignment and inference speed, overcoming the latency and coherence issues found in sequential (video-then-audio) generation pipelines.

How it works (AI summary)

The model employs an asymmetric dual-stream Diffusion Transformer (DiT) consisting of a 14B parameter video stream and a 5B parameter audio stream. These streams process decoupled latents from modality-specific VAEs and communicate via bidirectional cross-attention layers and cross-modality AdaLN for timestep conditioning. Text conditioning is handled by a Gemma 3 encoder using a multi-layer feature extractor and 'thinking tokens' to enhance prompt adherence.