Skip to content
AI-generated research idea🟢Easy🔧Minor TweakRelated work found

Instruction-Controllable VLMs for Strict Formatting via Preference Optimization

Fine-tune open-source vision-language models using Direct Preference Optimization on a format-compliance dataset to strictly eliminate verbose explanations and enforce exact prompt output formats.

Time

2 weeks

Compute level

Multiple GPUs

Resources

8x NVIDIA A100 GPUs for DPO training

What the paper used

Qwen3-VL-8B trained for one epoch using a global batch size of 2048

Prerequisites

reinforcement learning

Scope

modify existing code

The Gap

Proprietary models like Gemini-3-Pro and Claude-Sonnet-4.5 struggled to output answers in the exact format required by the benchmark prompts, often producing verbose explanations that required a secondary LLM for parsing.

“Although the models are required to produce final answers in a specified format, they frequently generate uncontrolled intermediate reasoning or extended explanations, violating the output constraints.”
From the paper, Page 10. Matched against the PDF text.

Limitation

High-capacity VLMs suffer from an over-helpfulness alignment tax, rendering them brittle for automated metric evaluation frameworks that require strict syntax.

Proposed Solution

Apply Direct Preference Optimization (DPO) targeting format adherence. Create a lightweight dataset of format-compliant versus overly-verbose responses and fine-tune an open-source model to strictly penalize extraneous tokens while preserving spatial reasoning accuracy.

Why This Matters

Improves the deployment efficiency and API cost of VLMs in programmatic pipelines, removing the latency and unreliability of secondary parser LLMs.

How to Get Started

How to test it: Hypothesis: DPO targeting format adherence reduces extraneous tokens while maintaining spatial accuracy. Baseline: SFT-adapted Qwen3-VL-8B. Metric: Exact format compliance rate and spatial reasoning accuracy. Stop if format compliance does not reach 95%.

  1. Collect or synthesize a small training set of format-compliant versus verbose responses using an open-weights VLM baseline like LLaVA or Qwen-VL.
  2. Set up the Direct Preference Optimization training pipeline using the TRL library on a standard GPU.
  3. Establish an automated evaluation script to measure format adherence and spatial reasoning accuracy.
  4. Fine-tune the model and evaluate against proprietary baselines on targeted strict-formatting benchmarks.

Code linked to this paper on Hugging Face: github.com/Visionary-Laboratory/CourtSI

Suggested by AI. Adapt it to your setup.

Has This Been Done?

Related work found

Studies like Uni-DPO and VIVA explore preference optimization and reward mechanisms for large language models and multimodal tasks. However, the specific application of direct preference optimization for strict format adherence and penalty of extraneous tokens in spatial VLMs is not directly evaluated by these works.

We searched OpenAlex and arXiv for “Fine-tune open-source vision-language models using Direct Preference Optimization on a format-compliance dataset to strictly eliminate verbose explanations and enforce exact prompt output formats.” and “Instruction-Controllable VLMs for Strict Formatting via Preference Optimization” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

Stepping VLMs onto the Court: Benchmarking Spatial Intelligence in Sports

Yuchen Yang, Yuqing Shao, Duxiu Huang +11 more · 2026

This paper introduces CourtSI, the first large-scale spatial intelligence dataset and benchmark tailored for sports scenarios to evaluate Vision-Language Models (VLMs). By leveraging well-defined court geometries, the authors reconstruct 3D sports scenes to generate over 1 million metric-aware QA pairs, revealing significant gaps in current VLMs' spatial reasoning capabilities.

Related Ideas

Is this a good idea?

Share onXLinkedIn