Self-Supervised Geometric Distillation for Long-Horizon Data Walls
Harvest unlabelled video tours using an ensemble of foundation models to generate pseudo-ground-truth trajectories and scale-adjusted depth maps for pre-training geometry compression.
Time
Compute level
Resources
What the paper used
Prerequisites
Scope
The Gap
The architecture relies heavily on heavily weighted large-scale synthetic navigation datasets (like TartanAirV2) during curriculum training to learn effective geometry compression. The authors explicitly note a 'data wall' because real-world, minute-long video sequences with high-quality dense 3D ground truth are exceptionally rare.
“To overcome this data wall without waiting for the curation of massive long-horizon datasets with the requisite diversity and accuracy, we argue that end-to-end chunk-wise processing is a practical and effective strategy.”
Limitation
The model's generalization to in-the-wild scenarios is bottlenecked by the lack of diverse, real-world, long-horizon training data with dense 3D annotations.
Proposed Solution
Develop a self-supervised or pseudo-labeling pipeline to harvest massive unlabelled YouTube driving or walking tours. Use an ensemble of off-the-shelf foundation models (e.g., DepthAnything combined with robust SLAM/COLMAP) to generate reliable pseudo-ground-truth trajectories and scale-adjusted depth maps, then use this to pre-train the TTT memory module.
The paper’s authors list this direction as future work, so they may already be pursuing it.
Why This Matters
Breaking the data wall through self-supervised data curation will drastically improve the model's zero-shot generalization to diverse real-world environments without requiring expensive hardware-based data collection.
How to Get Started
How to test it: We hypothesize that pre-training on self-supervised pseudo-labeled YouTube tours improves in-the-wild generalization. We will measure ATE on VBR compared to the baseline trained on standard datasets, stopping if the error reduction is less than 5%.
- Collect a large unlabelled video corpus from YouTube driving and walking datasets.
- Set up an off-the-shelf perception ensemble combining DepthAnything with COLMAP or robust SLAM systems.
- Design a pseudo-labeling filtration pipeline to filter out low-confidence trajectories and inaccurate depth estimations.
- Implement a pre-training script using the generated pseudo-dataset to update the memory module weights.
Code linked to this paper on Hugging Face: github.com/Junyi42/LoGeR
Suggested by AI. Adapt it to your setup.
Has This Been Done?
GLaD investigates geometric latent distillation for vision models, and HorizonDrive explores self-corrective models for long-horizon driving. They do not specifically cover harvesting unlabelled driving tours using depth estimators and SLAM to pre-train a TTT memory module.
- GLaD: Geometric Latent Distillation for Vision-Language-Action Models · 2025 · 21 citations
- HorizonDrive: Self-Corrective Autoregressive World Model for Long-horizon Driving Simulation · 2026 · 7 citations
We searched OpenAlex and arXiv for “Harvest unlabelled video tours using an ensemble of foundation models to generate pseudo-ground-truth trajectories and scale-adjusted depth maps for pre-training geometry compression.” and “Self-Supervised Geometric Distillation for Long-Horizon Data Walls” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
LoGeR: Long-Context Geometric Reconstruction with Hybrid MemoryJunyi Zhang, Charles Herrmann, Junhwa Hur +5 more · 2026
This paper introduces LoGeR, a novel feedforward geometric foundation model designed for long-context dense 3D reconstruction. It overcomes the quadratic complexity of global attention and sequence length limitations by utilizing a chunk-wise processing architecture with a hybrid memory system, achieving state-of-the-art results on extremely long video sequences.
Similar Idea From 1 Other Paper
- Scaling Spatial Test-Time Training to In-the-Wild Videos via Self-Supervised Depth Estimation · Spatial-TTT: Streaming Visual-based Spatial Intelligence with Test-Time Training
Related Ideas
- Adaptive Reference-Free Extrapolation for Video World Models under Sparse GroundingGrounding World Simulation Models in a Real-World Metropolis🟠Medium
- Distilling Generative Video Depth Priors into Lightweight Linear-Complexity ModelsDVD: Deterministic Video Depth Estimation with Generative Priors🔴Hard
- 4D Spatiotemporal Pretraining for Dynamic ScenesUtonia: Toward One Encoder for All Point Clouds🔴Hard
- Unsupervised Motion Disambiguation via Semantic Scene Priorsπ^4: A Foundation Model for 4D Scene Geometry and Motion🟠Medium
- Hierarchical Decay Mechanisms for Infinite-Context MemorytttLRM: Test-Time Training for Long Context and Autoregressive 3D Reconstruction🟡Fairly easy