Skip to content
AI-generated research idea🔴Hard🔬Significant ResearchClosely matching work found

Extending Spatial Curation to Dynamic Scenes via 4D Representation

Incorporate 4D Gaussian Splatting and dynamic deformation fields to separate static backgrounds from moving objects in dynamic indoor environments.

Time

1-2 weeks

Compute level

Large cluster

Resources

32 NVIDIA H800 GPUs (80GB) for cluster-wide dynamic scene optimization and training.

What the paper used

12K optimized 3DGS scenes from ScanNet, ScanNet++, and DL3DV-10K

Prerequisites

dynamic scene reconstruction

Scope

new model architecture

The Gap

The current 2D-to-3D lifting and scene-level refinement stages assume a perfectly static indoor environment, relying heavily on standard Structure-from-Motion (SfM) and static 3DGS.

“The pipeline relies on multiple upstream components and per-scene optimization, which can be computationally expensive and may degrade under challenging videos (e.g., limited viewpoints, motion blur, heavy occlusion, or dynamic objects).”
From the paper, Impact Statement, page 13. Matched against the PDF text.

Limitation

The pipeline degrades significantly or fails entirely when applied to videos containing dynamic objects, heavy occlusions, or severe motion blur, limiting its use to static environment scans.

Proposed Solution

Incorporate 4D Gaussian Splatting or dynamic deformation fields into the Geometric Optimization phase. Modify the scene-level tracking to separate static background Gaussians from dynamic foreground Gaussians, enabling the extraction of temporally-aware 4D bounding boxes.

The paper’s authors list this direction as future work, so they may already be pursuing it.

Why This Matters

This allows the framework to process in-the-wild dynamic videos (such as YouTube vlogs or egocentric human-object interaction videos), massively expanding the domain diversity of the generated dataset.

How to Get Started

How to test it: We hypothesize that incorporating dynamic deformation fields will improve reconstruction fidelity on videos with moving objects. We will compare against static 3DGS baselines on dynamic subset sequences using 3D metric accuracy and rendering PSNR. The experiment will stop if dynamic tracking fails to exceed static baseline metrics on non-static scenes.

  1. Review existing dynamic 3D scene datasets containing moving foreground objects and complex backgrounds.
  2. Modify the Geometric Optimization phase to integrate 4D Gaussian Splatting and deformation field modules.
  3. Develop scene-level tracking algorithms to explicitly segregate static background Gaussians from dynamic foreground elements.
  4. Evaluate 4D bounding box extraction and tracking performance on synthetic and real-world dynamic sequences.

Code linked to this paper on Hugging Face: github.com/Visionary-Laboratory/holi-spatial

Suggested by AI. Adapt it to your setup.

Has This Been Done?

Closely matching work found

Papers such as 4D Gaussian Splatting: Modeling Dynamic Scenes with Native 4D Primitives and SDD-4DGS: Static-Dynamic Aware Decoupling in Gaussian Splatting for 4D Scene Reconstruction explicitly address separating static and dynamic components using 4D representations for scene reconstruction. They cover the core concepts of dynamic deformation fields and decoupled foreground-background modeling proposed in the idea.

We searched OpenAlex and arXiv for “Incorporate 4D Gaussian Splatting and dynamic deformation fields to separate static backgrounds from moving objects in dynamic indoor environments.” and “Extending Spatial Curation to Dynamic Scenes via 4D Representation” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

Holi-Spatial: Evolving Video Streams into Holistic 3D Spatial Intelligence

Yuanyuan Gao, Hao Li, Yifei Liu +14 more · 2026

This paper introduces Holi-Spatial, a fully automated pipeline designed to convert raw video streams into high-fidelity 3D spatial annotations without human intervention. The primary contribution is the creation of Holi-Spatial-4M, a massive, spatially-aware multimodal dataset that significantly enhances the spatial reasoning and 3D grounding capabilities of Vision-Language Models.

Related Ideas

Is this a good idea?

Share onXLinkedIn