Iterative Semantic Correction for Layout Analysis
Implement a feedback loop that uses semantic VLM token probabilities and perplexity spikes to iteratively correct and regraph layout errors from upstream detectors.
Source paper
Cheng Cui, Ting Sun, Suyin Liang, Tingquan Gao, Zelun Zhang, Jiaxuan Liu, Xueqing Wang, Changda Zhou, Hongen Liu, Manhui Lin, Yue Zhang, Yubo Zhang, Handong Zheng, Jing Zhang, Jun Zhang, Yi Liu, Dianhai Yu, Yanjun Ma
Generated by AI. The paper’s authors did not write these.
Implement a feedback loop that uses semantic VLM token probabilities and perplexity spikes to iteratively correct and regraph layout errors from upstream detectors.
Integrate a lightweight token-level retrieval-augmented generation module into low-parameter vision models to reduce hallucinations of rare entities without full retraining.
Add 2D-attention adapter layers to the decoder of vision-language models to preserve grid-like table topologies and improve complex table structure recognition.
The paper presents PaddleOCR-VL, a resource-efficient document parsing system built around a 0.9B parameter Vision-Language Model (VLM). By decoupling layout analysis (using PP-DocLayoutV2) from element recognition (using the VLM with a NaViT-style encoder and ERNIE-4.5-0.3B backbone), the authors achieve SOTA performance on multilingual document benchmarks while maintaining low latency suitable for deployment.
The system employs a two-stage pipeline: first, PP-DocLayoutV2 (based on RT-DETR and a pointer network) detects layout elements and reading order; second, PaddleOCR-VL-0.9B (combining a dynamic resolution visual encoder and a 0.3B LLM) performs fine-grained recognition of text, tables, formulas, and charts using instruction tuning on a massive dataset of synthetic and real-world samples.