Generative Synthesis of Hard Case Training Data
Synthesize hard training document images using generative models that specifically target identified failure modes like heavy watermarks and extreme aspect ratios, replacing heuristic mining.
Time
Compute level
Resources
Prerequisites
Scope
The Gap
Section 4.3 details the 'Iterative Mining via Inference Consistency' (IMIC) strategy, which identifies hard training samples by running inference three times on real data and measuring divergence.
Limitation
The current data engine is passive; it relies on the fortuitous existence of hard cases within the collected PDF pool. It is also computationally expensive (requiring 3x inference) and cannot produce data for edge cases that are theoretically possible but absent from the raw corpus.
Proposed Solution
Develop a generative framework (e.g., using diffusion models or programmatic rendering) trained to synthesize document images that mimic the specific failure modes identified by IMIC (e.g., extreme aspect ratios, heavy watermarks, nested tables). This moves from 'mining' hard cases to 'creating' them.
Why This Matters
Proactively generating difficult training data allows the model to become robust against rare edge cases much faster than waiting to encounter them in the wild, breaking the dependency on massive raw data collection.
How to Get Started
- Run inference three times on a baseline document dataset to record divergence maps and identify failure modes.
- Collect and annotate a specialized training set containing extreme aspect ratios and degraded backgrounds.
- Fine-tune a Stable Diffusion model using LoRA to generate targeted synthetic document anomalies.
- Train the downstream OCR pipeline on the generated synthetic data and measure robustness gains on real benchmarks.
Code linked to this paper on Hugging Face: github.com/opendatalab/MinerU
Suggested by AI. Adapt it to your setup.
Has This Been Done?
The provided results cover general generative synthesis techniques like Data synthesis based on generative adversarial networks and training methods, but none of them address the specific generation of hard document cases targeting failure modes like extreme aspect ratios or heavy watermarks.
- Data synthesis based on generative adversarial networks · 2018 · 784 citations
- Training Generative Adversarial Networks with Limited Data · 2020 · 2643 citations
- Texture Synthesis with Spatial Generative Adversarial Networks · 2016 · 258 citations
We searched OpenAlex for “Synthesize hard training document images using generative models that specifically target identified failure modes like heavy watermarks and extreme aspect ratios, replacing heuristic mining.” and “Generative Synthesis of Hard Case Training Data” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document ParsingJunbo Niu, Zheng Liu, Zhuangcheng Gu +58 more · 2025
MinerU2.5 is a 1.2B-parameter vision-language model designed for efficient, high-resolution document parsing. It achieves state-of-the-art accuracy by decoupling the process into two stages: global layout analysis on a downsampled image and fine-grained content recognition on native-resolution crops, supported by a sophisticated data engine that mines 'hard cases' via inference consistency.
Related Ideas
- Self-Supervised Consistency Learning for Real-World Robustnessdots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Model🟠Medium
- Stabilized Online Hard Example Mining via Momentum-based Difficulty EstimationIn Pursuit of Pixel Supervision for Visual Pre-training🟡Fairly easy
- End-to-End Joint Optimization of MLLM and MMDiTUniVideo: Unified Understanding, Generation, and Editing for Videos🔴Hard
- Learnable Neural Degradation SimulationNeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos🟠Medium
- Direct Visual-to-Graph Generation for Irregular LayoutsRAG-Anything: All-in-One RAG Framework🟠Medium