Dynamic Content-Aware Compression for High-Density Layouts
Eliminate rigid image tiling for dense layouts by using a dynamic token allocator that assigns more latent tokens to high-entropy regions.
Source paper
Generated by AI. The paper’s authors did not write these.
Eliminate rigid image tiling for dense layouts by using a dynamic token allocator that assigns more latent tokens to high-entropy regions.
Evaluate conversational agents on optically-rendered dialogue histories by rendering chat logs as images for downstream visual OCR ingestion.
Add a graph-matching loss to the geometric parsing head so the model penalizes invalid topological connections instead of purely sequential token errors.
DeepSeek-OCR proposes a novel Vision-Language Model framework designed to treat optical character recognition (OCR) as a method for compressing long textual contexts into efficient vision tokens. By utilizing a specialized DeepEncoder (combining SAM for local attention and CLIP for global attention) connected to a Mixture-of-Experts decoder, the model achieves state-of-the-art OCR performance on the OmniDocBench while using significantly fewer visual tokens (64-100) compared to existing solutions.
The architecture features a 'DeepEncoder' that processes high-resolution images using a dual-pathway: a window-attention based SAM encoder and a global-attention CLIP encoder, bridged by a 16x convolutional compression layer to downsample vision tokens. These compressed tokens are fed into a DeepSeek3B-MoE decoder trained via next-token prediction on a diverse dataset of documents, charts, and synthetic formulas.