Skip to content
AI-generated research idea🟡Fairly easy🔧Minor Tweak

Grammar-Constrained Decoding for Structured DocTags

Implement a grammar-constrained decoding layer using an FSM or CFG parser during inference to force language models to select tokens maintaining valid DocTags syntax.

Time

several weeks

Compute level

Single GPU or less

Resources

Standard GPU

Prerequisites

LLM decoding strategies, Finite State Machines (FSM) or Context-Free Grammars (CFG)

Scope

Modify inference code to include logit masking

The Gap

As described in Section 5.3 (Qualitative Results), the model relies on standard autoregressive generation which frequently results in malformed structures, such as missing closing tags or missing location identifiers.

Limitation

The model occasionally produces syntactically invalid markup and 'endless repetition' loops, which prevents downstream parsers from correctly rendering the converted document.

Proposed Solution

Implement a grammar-constrained decoding layer (e.g., using a FSM or CFG parser) during inference to force the LLM to only select tokens that maintain valid DocTags syntax, specifically enforcing balanced XML tags and coordinate-range validity for <loc_x> tags.

Why This Matters

Ensuring 100% syntactical validity is critical for production-grade document conversion where automated downstream processing is required.

How to Get Started

  1. Define the formal grammar for DocTags, focusing on balanced XML tags and coordinate-range validity.
  2. Integrate a finite state machine or context-free grammar parser into the generation loop of a target model.
  3. Validate the constrained decoder on a test set of malformed document generation tasks.
  4. Measure the reduction in syntax errors and structural failures compared to standard autoregressive generation.

Code linked to this paper on Hugging Face: github.com/docling-project/docling

Suggested by AI. Adapt it to your setup.

Has This Been Done?

Not checked against the literature yet.

Source Paper

SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion

Ahmed Nassar, Andres Marafioti, Matteo Omenetti +10 more · 2025

SmolDocling is a 256M parameter vision-language model designed for efficient, end-to-end document conversion, mapping page images directly to a structured 'DocTags' markup format. It significantly reduces computational overhead compared to larger models while maintaining robust performance across diverse document elements like tables, charts, and equations.

Related Ideas

Is this a good idea?

Share onXLinkedIn