Fully Automated Metric Depth Estimation for Dynamic Small Objects
Develop a physics-informed tracking module combining small-object detection with dynamics constraints to automate 3D depth estimation for fast-moving items.
Source paper
Yuchen Yang, Yuqing Shao, Duxiu Huang, Linfeng Dong, Yifei Liu, Suixin Tang, Xiang Zhou, Yuanyuan Gao, Wei Wang, Yue Zhou, Xue Yang, Yanfeng Wang, Xiao Sun, Zhihang Zhong
Generated by AI. The paper’s authors did not write these.
Develop a physics-informed tracking module combining small-object detection with dynamics constraints to automate 3D depth estimation for fast-moving items.
Inject camera intrinsics and extrinsics directly into vision-language model encoders using geometric adapters to resolve perspective ambiguity in spatial reasoning.
Implement a domain-agnostic spatial pre-training objective using abstract geometric graphs to improve zero-shot cross-sport spatial generalization.
Fine-tune open-source vision-language models using Direct Preference Optimization on a format-compliance dataset to strictly eliminate verbose explanations and enforce exact prompt output formats.
This paper introduces CourtSI, the first large-scale spatial intelligence dataset and benchmark tailored for sports scenarios to evaluate Vision-Language Models (VLMs). By leveraging well-defined court geometries, the authors reconstruct 3D sports scenes to generate over 1 million metric-aware QA pairs, revealing significant gaps in current VLMs' spatial reasoning capabilities.
The methodology uses a semi-automatic data engine leveraging fixed court dimensions for metric-aware camera calibration via a Perspective-n-Point (PnP) solver. This is combined with PromptHMR for 3D human mesh recovery and human-in-the-loop annotations to establish a unified world coordinate system, enabling the programmatic generation of spatial QA pairs.