Skip to content
AI-generated research idea🟡Fairly easy💡Moderate InnovationNo close match found

Domain-Agnostic Pre-training for Zero-Shot Cross-Sport Spatial Generalization

Implement a domain-agnostic spatial pre-training objective using abstract geometric graphs to improve zero-shot cross-sport spatial generalization.

Time

4 weeks

Compute level

Large cluster

Resources

16x NVIDIA A100 GPUs for pre-training and fine-tuning

What the paper used

1M training QA pairs across badminton, tennis, and table tennis

Prerequisites

basic ML knowledge

Scope

modify existing code

The Gap

The Qwen3-VL-8B model fine-tuned on CourtSI showed only limited improvements when evaluated on CourtSI-Ext (pickleball), specifically struggling with localization in the unseen sport.

“Furthermore, models trained on existing spatial intelligence benchmarks generalize poorly to CourtSI-Bench, suggesting that current datasets fail to sufficiently capture the challenges posed by dynamic sports scenarios.”
From the paper, Page 3. Matched against the PDF text.

Limitation

The spatial reasoning capabilities learned by the VLM overfit to the specific court semantics and geometries of the training sports, failing to generalize to unseen, structurally similar domains.

Proposed Solution

Implement a domain-agnostic spatial pre-training objective. Instead of QA pairs tied to specific sports terms, fine-tune the model using abstract geometric representations (e.g., replacing court textures with grid lines via segmentation masks) or use contrastive learning to align abstract spatial graphs with textual descriptions.

Why This Matters

This yields a universally applicable spatial VLM capable of out-of-the-box 3D reasoning for any new environment without requiring costly domain-specific 3D data collection.

How to Get Started

How to test it: Hypothesis: Abstract geometric pre-training improves cross-sport spatial generalization. Baseline: Base Qwen3-VL-8B fine-tuned on CourtSI. Metric: Accuracy on CourtSI-Ext (pickleball). Stop if accuracy on unseen sport drops below baseline levels.

  1. Prepare the Qwen3-VL-8B model baseline and evaluate zero-shot performance on the CourtSI-Ext pickleball benchmark.
  2. Create an abstract spatial dataset by replacing specific court textures with standardized grid lines and segmentation masks.
  3. Design a contrastive pre-training objective aligning the abstract spatial layouts with descriptive spatial text.
  4. Fine-tune the model on the abstracted spatial pre-training task before testing zero-shot transfer to unseen sports.

Code linked to this paper on Hugging Face: github.com/Visionary-Laboratory/CourtSI

Suggested by AI. Adapt it to your setup.

Has This Been Done?

No close match in 10 results

The listed literature contains general sports deep learning surveys like A Survey of Deep Learning in Sports Applications and transfer learning works like OmniTraj, but none propose domain-agnostic pre-training using abstract geometric representations and grid masks for zero-shot cross-sport generalization.

We searched OpenAlex and arXiv for “Implement a domain-agnostic spatial pre-training objective using abstract geometric graphs to improve zero-shot cross-sport spatial generalization.” and “Domain-Agnostic Pre-training for Zero-Shot Cross-Sport Spatial Generalization” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

Stepping VLMs onto the Court: Benchmarking Spatial Intelligence in Sports

Yuchen Yang, Yuqing Shao, Duxiu Huang +11 more · 2026

This paper introduces CourtSI, the first large-scale spatial intelligence dataset and benchmark tailored for sports scenarios to evaluate Vision-Language Models (VLMs). By leveraging well-defined court geometries, the authors reconstruct 3D sports scenes to generate over 1 million metric-aware QA pairs, revealing significant gaps in current VLMs' spatial reasoning capabilities.

Related Ideas

Is this a good idea?

Share onXLinkedIn