Skip to content
AI-generated research idea🟡Fairly easy🔧Minor TweakNo close match found

Vertical and non-Manhattan Layout Parsing

Augment synthetic document rendering pipelines with vertical layouts so vision models learn to parse ancient and non-Manhattan scripts.

Time

several weeks

Compute level

Single GPU or less

Resources

Standard GPU, specialized datasets (e.g., ancient manuscripts)

Prerequisites

basic ML knowledge, data engineering

Scope

large-scale data collection

The Gap

The model's Stage 1 layout analysis and Stage 2 text recognition are trained on standard horizontal text. The paper admits 'Dolphin primarily supports documents with standard horizontal text layout, showing limited capability in parsing vertical text like ancient manuscripts.'

Limitation

The current reading-order prediction and cropping mechanisms assume a left-to-right, top-to-bottom flow, making the model fail on vertical text (Asian scripts) or complex non-Manhattan layouts (artistic magazines).

Proposed Solution

Augment the synthetic data generation pipeline to render documents with vertical text flow and irregular layouts. Retrain the Stage 1 position embeddings and Stage 2 recognition module to handle vertical aspect ratios and top-to-bottom reading directions.

Why This Matters

Crucial for digitizing historical archives (e.g., ancient Chinese/Japanese manuscripts) and processing diverse commercial designs, expanding the model's utility beyond standard academic/business documents.

How to Get Started

  1. Build a synthetic document rendering pipeline capable of generating top-to-bottom vertical text flows.
  2. Collect and annotate a small benchmark dataset of ancient manuscripts containing irregular layouts.
  3. Retrain the Stage 1 position embeddings to handle non-standard vertical aspect ratios and reading directions.
  4. Fine-tune the Stage 2 recognition module on the augmented vertical text dataset.

Code linked to this paper on Hugging Face: github.com/bytedance/dolphin

Suggested by AI. Adapt it to your setup.

Has This Been Done?

No close match in 6 results

The listed works all focus on 3D room layout reconstruction and Manhattan world inference from images or point clouds. None of them discuss document layout parsing, synthetic data generation for text, or vertical text flow and reading directions.

We searched OpenAlex for “Augment synthetic document rendering pipelines with vertical layouts so vision models learn to parse ancient and non-Manhattan scripts.” and “Vertical and non-Manhattan Layout Parsing” and an AI compared the idea with the top 6 results. A small search can miss relevant work.

Source Paper

Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting

Hao Feng, Shu Wei, Xiang Fei +10 more · 2025

This paper introduces Dolphin, a multimodal document image parsing model that employs an "analyze-then-parse" paradigm to extract structured content from documents. By first identifying layout elements (e.g., paragraphs, tables) and their reading order, and then using these elements as anchors for parallel content parsing with specialized prompts, Dolphin achieves state-of-the-art performance and high efficiency compared to existing end-to-end VLMs.

Related Ideas

Is this a good idea?

Share onXLinkedIn