Generative Synthesis of Articulated Kinematic Chains
Generate articulated 3D kinematic structures directly by predicting joint types and movement axes from part-segmented geometries.
Source paper
Nicholas Pfaff, Thomas Cohn, Sergey Zakharov, Rick Cory, Russ Tedrake
Generated by AI. The paper’s authors did not write these.
Generate articulated 3D kinematic structures directly by predicting joint types and movement axes from part-segmented geometries.
Integrate soft-body physics solvers into the pipeline to simulate realistic thin coverings like rugs and tablecloths with collision geometry and physical folds.
Establish an adversarial feedback loop between the robot's success metrics and the scene generator to produce challenging curricula that improve policy generalization.
Distill the expensive iterative Designer-Critic agent loop into a fast, non-iterative Diffusion Transformer conditioned on layout graphs.
SceneSmith is a hierarchical, agentic framework that utilizes Large Language Models (LLMs) and Vision-Language Models (VLMs) to generate diverse, simulation-ready indoor environments from text prompts. By decomposing scene generation into sequential stages (layout, furniture, wall/ceiling objects, manipulands) managed by a Designer-Critic-Orchestrator agent trio, the system produces physically valid scenes with dense clutter and articulated objects suitable for robotic training.
The framework employs a tree-structured construction process where a 'Designer' agent proposes asset placements and a 'Critic' agent evaluates them against the text prompt and physical heuristics. It integrates a hybrid asset pipeline: static objects are generated via text-to-3D (using SAM3D), articulated objects are retrieved from a library (ArtVIP), and physics properties (mass, friction) are estimated by VLMs. The final output is processed through a physics engine (Drake) to resolve collisions and ensure static stability.