Structured Item-by-Item Multimodal Verification for Iterative 4D Scene Coding
Verify generated 3D scene renders item-by-item against scene briefs using structured multimodal checklists so coding agents catch and fix spatial and camera errors.
Source paper
Yuzhe Chen, Muxi Diao, Lele Yang, Wenbin Xu, Xuyang Liu, Qile Su, Changyi Liu, Jiankang Chen, Kaiyu Jiang, Haoyu Zhang, Hongyi Fu, Jianxiong Wang, Tianke Zhang, Haonan Fan, Yingxin Li, Bingyi Li, Xu Wang, Xuanyu Zheng, Bin Wen, Tingting Gao, Wentao Zhang, Jiale Cao
Generated by AI. The paper’s authors did not write these.
Verify generated 3D scene renders item-by-item against scene briefs using structured multimodal checklists so coding agents catch and fix spatial and camera errors.
Equip scene-coding agents with explicit 3D depth, camera-pose, and human-motion estimators so they reconstruct geometry from reference media instead of guessing from raw pixels.
Enforce explicit geometric constraints and parameterized camera rigs during scene coding so agents execute rare angles, orbital moves, and exact shot counts instead of defaulting to common habits.
Benchmark how strongly 4D scene-accuracy scores predict final synthesized video quality across diverse agents and generative renderers to validate proxy-level evaluation.
Incorporate ground-plane contact, center-of-mass support, and joint-pose checks into deterministic scene evaluators to catch floating characters and missed body-posture events.
The paper introduces VibePrevisBench, a benchmark of 242 multi-shot, multi-person tasks for evaluating how well coding agents write executable Blender scripts to generate 4D scenes and graybox proxy videos from scripts, photos, frame continuations, and reference videos. Deliveries are scored via a deterministic Rule-based Scene Check across nine spatial, temporal, and physical dimensions and a VLM-based Screen Check assessing on-screen recognizability and story readability. Evaluating 11 frontier coding agents reveals that visual-spatial understanding separates models, all agents fall back on common camera defaults for challenging requests, and agents fail to fix errors when inspecting their own renders.
Each agent operates inside an 8-CPU, GPU-less Blender 5.2 sandbox with a 4-hour budget per task, iteratively writing Python code, rendering preview frames, inspecting outputs, and tagging entities and places. Submitted programs are re-executed from scratch in a clean sandbox to export frame-by-frame 4D bounding boxes, camera trajectories, and cut timestamps, which are evaluated deterministically against hidden ground-truth facts (from scene specs, 3D scan datasets, or an automated 4D reconstruction pipeline using SAM 3, MapAnything/VGGT, and GVHMR) alongside a two-turn Gemini 4 VLM judge watching the rendered proxy video at 2 fps.