Variable-Visit Training and Depth-Relative Query Routing for Test-Time Loop Scaling
Train parameter-shared transformers with randomized loop counts and depth-relative attention queries so extra inference iterations improve predictions instead of degrading accuracy.
Time
Compute level
Resources
What the paper used
Prerequisites
Scope
The Gap
In Section 5.2.3 and Appendix H, Loopback is trained at a fixed loop visit count R (e.g., R=4 at 191M, R=2 at 374M-3B) with application-specific learned queries q_s. When extra visits are added at inference without retraining by reusing the last trained visit's queries, validation loss increases across all tested scales (Fig. 6a-c: +0.060 at 191M with 8 visits, +0.074 at 374M with 4 visits, +0.028 at 1.4B with 4 visits), and 3B zero-shot gains are inconsistent across tasks (Table 14: 5 of 13 tasks drop while RTE contributes 0.56 of the +0.72 point mean increase).
“Shared weights allow extra visits at inference, but excessive recurrence can degrade predictions, a problem known as overthinking (Bansal et al., 2022).”
Limitation
Fixed-visit training and step-indexed queries prevent Loopback from generalizing reliably to additional loop iterations at test time, failing to turn extra compute into consistent validation-loss or downstream-task improvements.
Proposed Solution
Train with stochastic visit counts R sampled per batch or step from [R_min, R_max] (applying loss at the sampled exit or via depth-wise self-distillation) and replace fixed-index queries q_s with queries generated from a shared base embedding plus sinusoidal or normalized-depth embeddings of (r/R, k/K) conditioned on the current hidden state. Combine this with an inference-time early-exit or KL-convergence criterion so tokens stop looping before overthinking degrades representations.
The paper’s authors list this direction as future work, so they may already be pursuing it.
Why This Matters
Enabling monotonic or controllable quality improvements with extra test-time iterations would unlock genuine latent-reasoning compute scaling without needing separate checkpoints per depth.
How to Get Started
How to test it: Hypothesis: variable-visit training with depth-relative queries eliminates validation-loss degradation up to 2x the nominal visit count while matching fixed-visit loss at the base depth. Evaluate 191M (R in 2..8) and 374M (R in 1..4) models on ClimbMix validation loss and the 13-task lm-evaluation-harness suite against fixed-visit Loopback; stop if base-depth validation loss worsens by more than 0.005 nats or extra visits still monotonically increase loss.
- Reproduce the 191M [2L -> 4L (x4) -> 2L] and 374M [2L -> 10L (x2) -> 2L] Loopback baselines and Figure 6a-b extra-visit curves.
- Replace per-step queries q_s in the Loopback reader with a lightweight MLP mapping continuous depth features (r/R, k/K) to query vectors in R^d.
- Implement per-step uniform sampling of R during 20-TPP ClimbMix training, ensuring FSDP gradient synchronization across dynamic loop lengths.
- Evaluate trained checkpoints across R = 2 to 8 (191M) and R = 2 to 4 (374M) on ClimbMix validation loss and all 13 zero-shot benchmarks.
Suggested by AI. Adapt it to your setup.
Has This Been Done?
Scaling Test-Time Compute with Latent Reasoning [2] and LatentLoop [9] discuss test-time compute and variable depth, and some papers like [4] investigate truncated backpropagation. However, the specific combination of training with stochastic visit counts alongside a sinusoidal depth-relative query routing mechanism conditioned on hidden states is not explicitly detailed in these works.
- Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach · 2025 · 558 citations
- Truncate or Not? A Scaling Study of Looped Transformers with Truncated Backpropagation · 2026 · 1 citations
- LatentLoop: Scaling Looped MoE Models with Asymmetric Compute Allocation · 2026 · 0 citations
We searched OpenAlex and arXiv for “looped transformer variable depth training test-time scaling overthinking” and “Variable-Visit Training and Depth-Relative Query Routing for Test-Time Loop Scaling” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
Loopback Transformer: Towards a Unified View of Residual Looped TransformersXuan Ouyang, Zefan Cai, Chenlong Zhang +3 more · 2026
This paper introduces a depth-memory framework that unifies residual connections in looped Transformers with linear attention and recurrent memory via write, transport, and read operations. Building on this analysis, the authors propose the Loopback Transformer, which achieves lower validation loss and higher thirteen-task zero-shot accuracy than depth-matched standard Transformers across 191M to 3B scales while using 40-75% of their Transformer parameters.
Related Ideas
- Feedback-Loop Retrieval VerificationSimpleMem: Efficient Lifelong Memory for LLM Agents🟢Easy
- Dynamic Working-Memory Pruning for Search Agents to Overcome Reinforcement Learning SaturationK2-Horizon: A Frontier Fleet of Open Source Language Models🟡Fairly easy
- Difficulty- and Saturation-Aware Adaptive Task-Level Recursion BudgetingHarness^2: Recursive Agent Harnessing for an Open World🟡Fairly easy
- Multi-Shape Generalization and End-to-End Workload Evaluation for NPU Kernel OptimizationAscendGYM: A Cost-Aware Kernel Optimization Benchmark for Real-World Workloads on Ascend NPUs🟠Medium
- Mitigating Few-Shot Overfitting via Dynamic Sample RotationSelf-Supervised Prompt Optimization🟢Easy