Global Context Injection for Local Crop Decoding
Condition local document crop recognition on a compressed global semantic vector extracted from the vision encoder to improve layout-aware document understanding without high-resolution full-image processing.
Time
Compute level
Resources
Prerequisites
Scope
The Gap
Section 3.2 describes Stage II as performing recognition on 'native-resolution crops extracted from the original image'. These crops are fed into the VLM independently.
Limitation
By processing crops in isolation, the decoder loses access to global semantic context. This hinders performance in ambiguous scenarios (e.g., deciphering degraded text that requires topic knowledge) and disrupts continuity when sentences or tables are split across multiple crops (e.g., multi-column text).
Proposed Solution
Modify the architecture to extract a compressed global semantic vector from the Stage 1 vision encoder (NaViT). Inject this vector as a prefix token or via cross-attention into the Stage 2 decoder. This conditions the local crop generation on the global document context without processing the full image at high resolution.
Why This Matters
Restoring global context improves optical character recognition (OCR) robustness in noisy conditions and ensures semantic consistency (e.g., correct acronym expansion) across the entire document.
How to Get Started
- Implement a projection layer on the NaViT vision encoder to output a fixed-size global semantic vector.
- Modify the Stage 2 decoder architecture to accept the global vector via cross-attention layers.
- Fine-tune the modified model on document parsing benchmarks like DocVQA using standard OCR datasets.
- Evaluate token efficiency and extraction accuracy against the baseline independent-crop architecture.
Code linked to this paper on Hugging Face: github.com/opendatalab/MinerU
Suggested by AI. Adapt it to your setup.
Has This Been Done?
None of the listed papers address injecting a compressed global semantic vector from a Stage 1 vision encoder into a Stage 2 decoder for local crop generation in document understanding. While works like Beyond the Grid and UniDoc study document representations and global-local methods appear in other domains, the specific architecture proposed here is not covered by the listed results.
- Beyond the Grid: Layout-Informed Multi-Vector Retrieval with Parsed Visual Document Representations · 2026 · 7 citations
- UniDoc: Towards Unified Thinking in Documents · 2025 · 8 citations
- Unifying Global-Local Representations in Salient Object Detection With Transformers · 2024 · 66 citations
We searched OpenAlex and arXiv for “Condition local document crop recognition on a compressed global semantic vector extracted from the vision encoder to improve layout-aware document understanding without high-resolution full-image processing.” and “Global Context Injection for Local Crop Decoding” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document ParsingJunbo Niu, Zheng Liu, Zhuangcheng Gu +58 more · 2025
MinerU2.5 is a 1.2B-parameter vision-language model designed for efficient, high-resolution document parsing. It achieves state-of-the-art accuracy by decoupling the process into two stages: global layout analysis on a downsampled image and fine-grained content recognition on native-resolution crops, supported by a sophisticated data engine that mines 'hard cases' via inference consistency.
Related Ideas
- Multi-Granularity Visual Encoding for Small-Scale Document ElementsSmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion🟠Medium
- Multi-Scale Hierarchical Masking for Dense Prediction TasksIn Pursuit of Pixel Supervision for Visual Pre-training🟠Medium
- Bypassing Pixel Reconstruction for Efficient OCR-Centric Latent ProcessingCheers: Decoupling Patch Details from Semantic Representations Enables Unified Multimodal Comprehension and Generation🟠Medium
- Iterative Semantic Correction for Layout AnalysisPaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Model🟠Medium
- Entity-Disentangled Memory via SegmentationStoryMem: Multi-shot Long Video Storytelling with Memory🔴Hard