2D-Attentional Adapters for Complex Table Topology
Add 2D-attention adapter layers to the decoder of vision-language models to preserve grid-like table topologies and improve complex table structure recognition.
Time
Compute level
Resources
Prerequisites
Scope
The Gap
In Table 3, the model performs worse on English Table TEDS (88.0) than Chinese (92.14). The methodology (Section A.2) uses OTSL, which linearizes tables into token sequences. This linearization often struggles with spanning cells in English tables where aspect ratios differ significantly from training priors.
Limitation
The autoregressive serialization of tables (OTSL) loses inherent 2D spatial relationships, causing errors in complex spanning cells or nested headers common in English financial reports.
Proposed Solution
Introduce a 2D-Attention Adapter layer into the VLM decoder specifically for the 'Table Recognition' task. Instead of standard 1D causal attention, mask the attention mechanism to preserve the grid-like structure of the table (row/column alignment) based on the visual encoder's patch positions, enforcing structural consistency.
Why This Matters
Improving table structure recognition is critical for RAG applications where financial and scientific data must be extracted with 100% structural fidelity.
How to Get Started
- Inspect the attention mechanism within the VLM decoder used for table recognition tasks.
- Implement a 2D-attentional mask based on visual patch coordinates to restrict attention to valid row and column neighborhoods.
- Insert the adapter layers into the pre-existing model architecture and freeze the base weights if necessary.
- Train and evaluate on table recognition benchmarks such as TEDS to measure performance gains on English tables.
Suggested by AI. Adapt it to your setup.
Has This Been Done?
The listed literature contains works on visual table understanding like 'TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy' and general adapters, but none propose a 2D-Attention Adapter layer designed to mask attention mechanisms for enforcing structural grid alignment in table recognition tasks.
- TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy · 2024 · 135 citations
- HeBA: Heterogeneous Bottleneck Adapters for Robust Vision-Language Models · 2026 · 0 citations
We searched OpenAlex for “Add 2D-attention adapter layers to the decoder of vision-language models to preserve grid-like table topologies and improve complex table structure recognition.” and “2D-Attentional Adapters for Complex Table Topology” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language ModelCheng Cui, Ting Sun, Suyin Liang +15 more · 2025
The paper presents PaddleOCR-VL, a resource-efficient document parsing system built around a 0.9B parameter Vision-Language Model (VLM). By decoupling layout analysis (using PP-DocLayoutV2) from element recognition (using the VLM with a NaViT-style encoder and ERNIE-4.5-0.3B backbone), the authors achieve SOTA performance on multilingual document benchmarks while maintaining low latency suitable for deployment.
Related Ideas
- Generative Vision-Language Model for Table RecognitionMinerU: An Open-Source Solution for Precise Document Content Extraction🔴Hard
- Multi-Granularity Visual Encoding for Small-Scale Document ElementsSmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion🟠Medium
- Direct Visual-to-Graph Generation for Irregular LayoutsRAG-Anything: All-in-One RAG Framework🟠Medium
- Hierarchical Cell-Level Parallelism for Large TablesDolphin: Document Image Parsing via Heterogeneous Anchor Prompting🟠Medium
- Latent Visual-Audio Contrastive BindingDreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video Generation🟠Medium