Contact-Aware Physical Validity and Full-Body Posture Verification in 4D Scene Scorers
Incorporate ground-plane contact, center-of-mass support, and joint-pose checks into deterministic scene evaluators to catch floating characters and missed body-posture events.
Time
Compute level
Resources
What the paper used
Prerequisites
Scope
The Gap
In Section B.4, defect-injection stress tests on 12 high-scoring video deliveries show that the deterministic D1 Physical validity dimension misses floating people: lifting characters by 1 meter reduces the overall Rule-based Scene Check score by only 0.8 points. Additionally, Section C.3 notes that the current rule checks miss posture-driven events such as falls, depressing confirmed event completion rates.
“The physical-validity check misses floating people: lifting them by 1 m costs only 0.8.”
Limitation
The deterministic 4D scene evaluator relies primarily on coarse oriented bounding boxes and lacks fine-grained vertical ground-support checks and skeletal posture classification, allowing physically impossible floating subjects to go nearly unpenalized and full-body posture events to go undetected.
Proposed Solution
Extend the sandbox replay exporter to dump per-frame mesh/armature lowest-vertex elevations and key skeletal joint heights (pelvis, feet, head, hands) alongside oriented bounding boxes. Upgrade D1 Physical validity with ray-cast downward support verification against floor/furniture geometry (flagging unsupported foot-to-surface gaps during stance phases while permitting airborne intervals during annotated jumps) and add kinematic posture classifiers (standing, crouching, seated, lying/fallen) to B2 Events using relative pelvis-to-head vertical ratios and body-axis inclination.
Why This Matters
Closing the floating-subject and posture-event blind spots is expected to make deterministic 4D scene scoring significantly more faithful to physical plausibility and human action semantics without increasing VLM judge costs.
How to Get Started
How to test it: Test the upgraded D1 Physical validity and B2 Posture rules on the 12 defect-injected video deliveries from Section B.4 plus a set of fall/crouch/sit tasks, measuring score sensitivity to 1 m vertical lifts (targeting >= 8-point D1 penalty) and agreement with human annotations on posture events while keeping overall agent rankings consistent (Kendall tau >= 0.90) on clean deliveries. Stop if noisy mesh bounds in mannequin assets cause excessive false-positive floating penalties on valid seated or climbing scenes.
- Inspect the released replay exporter and D1 Physical validity scoring module in VibePrevisBench to identify how bounding-box floating and interpenetration are currently computed.
- Modify the Blender replay hook to extract per-frame lowest mesh vertices for each subject and perform downward BVH/ray-cast distance queries to supporting floor or prop meshes.
- Add kinematic posture-state features (vertical extent ratio and torso orientation) to detect falls, crouches, and seated poses in B2 Events.
- Re-run the Section B.4 defect-injection suite (including 1 m character lifts and swapped postures) to verify monotonic penalty scaling without penalizing valid jump intervals.
Suggested by AI. Adapt it to your setup.
Has This Been Done?
Although papers like CRISP and PhysCap address physical plausibility and motion capture, none of them focus on extending sandbox replay exporters with ray-cast downward support verification and kinematic posture classifiers for 4D scene scorers.
- PhysCap: physically plausible monocular 3D motion capture in real time · 2020 · 253 citations
- CRISP: Contact-Guided Real2Sim from Monocular Video with Planar Scene Primitives · 2025 · 7 citations
We searched OpenAlex and arXiv for “4D scene evaluation physical plausibility floating contact posture detection” and “Contact-Aware Physical Validity and Full-Body Posture Verification in 4D Scene Scorers” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
VibePrevisBench: Evaluating Agentic Coding for 4D Scene Generation in BlenderYuzhe Chen, Muxi Diao, Lele Yang +19 more · 2026
The paper introduces VibePrevisBench, a benchmark of 242 multi-shot, multi-person tasks for evaluating how well coding agents write executable Blender scripts to generate 4D scenes and graybox proxy videos from scripts, photos, frame continuations, and reference videos. Deliveries are scored via a deterministic Rule-based Scene Check across nine spatial, temporal, and physical dimensions and a VLM-based Screen Check assessing on-screen recognizability and story readability. Evaluating 11 frontier coding agents reveals that visual-spatial understanding separates models, all agents fall back on common camera defaults for challenging requests, and agents fail to fix errors when inspecting their own renders.
Related Ideas
- Force-Aware Policy for Contact-Rich InteractionsSimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulation🟡Fairly easy
- Feature-Parity Verification and Latency-Accuracy Co-Evaluation for Served Frontier VLAsScaleVLA: Scaling Vision-Language-Action Models to 552 Billion Parameters🟡Fairly easy
- Robust Visual-Tactile State Estimation for Occlusion HandlingSimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulation🟠Medium
- Conformal Multimodal Outcome Verification to Mitigate Premature Completion BiasEnvisioning Agentic Robotics: A New Era of Robotic Systems🟡Fairly easy
- Fully Automated Metric Depth Estimation for Dynamic Small ObjectsStepping VLMs onto the Court: Benchmarking Spatial Intelligence in Sports🔴Hard