Hierarchical Scene Graph Optimization for Non-Planar Placement
Extend 3D asset optimization frameworks with a hierarchical support-detection graph so child objects are positioned relative to their parent surfaces instead of the global ground plane.
Time
Compute level
Resources
Prerequisites
Scope
The Gap
The current 'PlanarModel' optimization strictly confines object movement to the global ground plane (4-DoF: translation x/z, rotation y, scale).
Limitation
The system fails to correctly place 'sub-objects' that rest on other surfaces (e.g., a lamp on a table or books on a shelf). These objects are either forced to the floor (clipping through furniture) or treated as floating, breaking physical plausibility.
Proposed Solution
Extend the optimization framework to support a hierarchical constraint graph. Implement a support-detection heuristic (e.g., using bounding box overlap and height analysis) to identify parent-child relationships. Modify the differentiable optimization to define the coordinate space of child objects relative to the top surface plane of their parent object, rather than the global ground plane.
Why This Matters
This would significantly increase the complexity of scenes the model can handle, allowing for realistic reconstruction of cluttered, lived-in environments which are standard in VFX and gaming.
How to Get Started
- Implement a bounding box overlap and height analysis heuristic to automatically detect parent-child support relationships in scenes.
- Modify the differentiable optimization pipeline to define coordinate spaces of child objects relative to top surface planes of parents.
- Test the hierarchical constraint system on multi-level furniture placement datasets.
- Compare placement accuracy and realism against the baseline planar optimization framework.
Code linked to this paper on Hugging Face: github.com/cgtuebingen/3D-RE-GEN
Suggested by AI. Adapt it to your setup.
Has This Been Done?
Papers such as Hierarchical Scene Graph Concepts for Spatial AI in Robotics and HSM: Hierarchical Scene Motifs for Multi-Scale Indoor Scene Generation discuss hierarchical scene graphs or motifs, but none of the listed works address differentiable scene graph optimization for non-planar placement with support-detection and relative child coordinate spaces.
- Hierarchical Scene Graph Concepts for Spatial AI in Robotics · 2026 · 0 citations
- HSM: Hierarchical Scene Motifs for Multi-Scale Indoor Scene Generation · 2026 · 33 citations
We searched OpenAlex for “Extend 3D asset optimization frameworks with a hierarchical support-detection graph so child objects are positioned relative to their parent surfaces instead of the global ground plane.” and “Hierarchical Scene Graph Optimization for Non-Planar Placement” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
3D-RE-GEN: 3D Reconstruction of Indoor Scenes with a Generative FrameworkTobias Sautter, Jan-Niklas Dihlmann, Hendrik P. A. Lensch · 2025
The paper introduces 3D-RE-GEN, a pipeline for reconstructing fully textured, artist-ready 3D indoor scenes from a single 2D image. It decomposes the scene into a clean background and individual objects, ensuring physical plausibility through a novel 4-DoF ground-plane alignment optimization and using a custom visual prompting strategy called 'Application-Querying' to generate high-quality assets for occluded regions.
Related Ideas
- Generative Synthesis of Articulated Kinematic ChainsSceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes🔴Hard
- Depth-Aware Field-of-View Memory Retrieval for Complex 3D LayoutsWorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representation🟡Fairly easy
- Multi-Agent World Modeling: Extending Camera-Conditioned DiTs to Dynamic EntitiesWorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representation⛔Very hard
- Dynamic Object Handling via Motion SegmentationWorldWarp: Propagating 3D Geometry with Asynchronous Video Diffusion🟡Fairly easy
- Hierarchical Anchor-Guided Batch Generation for 360-Degree Scene EditingGeometry-Guided Reinforcement Learning for Multi-view Consistent 3D Scene Editing🟡Fairly easy