Skip to content
AI-generated research idea🟡Fairly easy💡Moderate InnovationNo close match found

Closed-Loop Autoregressive Training to Mitigate Prerouter Feature Drift

Implement a closed-loop training curriculum inspired by DAgger to replace ground-truth execution one-hots with model predictions during prerouter distillation.

Time

1-2 weeks to implement scheduled sampling and multi-step rollout loss, run Phase 1-3 ablations on 8B and 35B, and evaluate on OpenCompass.

Compute level

Multiple GPUs

Resources

A multi-GPU training node (e.g., 4x-8x A100/H100 80GB) capable of running bf16 35B Phase 1-3 distillation over ~2M SFT and ~200k on-policy rows, an OpenCompass evaluation server, and a 24 GB Mac mini M4 Pro for decode verification.

What the paper used

Training runs on dequantized bf16 reconstructions of the 35B (Qwen3.6-35B-A3B) and 8B (Ling 3.0 tiny) models over ~2M teacher-generated rows in Phase 2 and ~200k rows in Phase 3; training GPU hardware and hours are not stated. Evaluation uses OpenCompass on a compute server and single-device inference on a Mac mini M4 Pro 24 GB and MacBook M2 16 GB.

Prerequisites

Transformer architectures, imitation learning (DAgger/scheduled sampling), MoE gating mechanisms.

Scope

Modify existing code in the Phase 1 training pipeline.

The Gap

In Section 3.2 ('Feature drift'), the authors state that a prerouter head's input features concatenate layer hidden states with two top-k one-hot expert vectors. During training (Phase 1), these vectors correspond to the base router's ground-truth selection, but during inference, the head receives its own prior predictions because the true router is replaced entirely.

“At training that is the base router’s selection; at decode it is the head’s own prediction, because the prediction replaced the router.”
From the paper, Section 3.2, page 6. Matched against the PDF text.

Limitation

The prerouter heads are trained under teacher-forcing conditions and are never exposed to their own routing errors during distillation, creating exposure bias and compounding prediction errors across sequential tokens.

Proposed Solution

Implement a closed-loop training curriculum inspired by DAgger or Scheduled Sampling for routing prediction. During Phase 1 distillation, gradually replace ground-truth execution one-hots with the model's own predicted top-k vectors with increasing probability over training steps. Furthermore, add a multi-step rollout loss that penalizes multi-token divergent trajectories, stabilizing prerouter representations against out-of-distribution drift.

Why This Matters

Mitigating compounding routing prediction drift improves routing fidelity over long generation horizons, reducing reliance on the recovery LoRA and narrowing the accuracy gap on long-context tasks.

How to Get Started

How to test it: Hypothesis: Closed-loop scheduled sampling and multi-step rollout loss during Phase 1 prerouter training reduce routing exposure bias and improve downstream accuracy. Baseline: Edge0-35B and Edge0-8B trained with standard teacher-forced Phase 1 followed by Phases 2-3; Metric: OpenCompass 5-benchmark average, AIME 2026 score, and multi-token prerouter agreement; Stop if after full Phase 1-3 training the OpenCompass average improves by less than 0.5 points over the baseline.

  1. Extract the Phase 1 distillation dataset containing layer hidden states and ground-truth expert routing vectors from an open-source teacher model like Qwen3.6-35B.
  2. Implement a scheduled sampling loop that progressively replaces ground-truth routing one-hots with the student model's own top-k predictions.
  3. Define a multi-step rollout loss function to penalize divergence in sequential token prediction paths.
  4. Evaluate prerouter accuracy and downstream task performance on single-node 8x A100/H100 setups against the standard non-adaptive baseline.

Code linked to this paper on Hugging Face: github.com/Edge0-AI/edge0

Suggested by AI. Adapt it to your setup.

Has This Been Done?

No close match in 5 results

None of the listed papers discuss autoregressive training curricula, DAgger, or Scheduled Sampling for routing prediction, as all retrieved results pertain to high-energy physics and astrophysics experiments.

We searched OpenAlex and arXiv for “Implement a closed-loop training curriculum inspired by DAgger to replace ground-truth execution one-hots with model predictions during prerouter distillation.” and “Closed-Loop Autoregressive Training to Mitigate Prerouter Feature Drift” and an AI compared the idea with the top 5 results. A small search can miss relevant work.

Source Paper

The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction

Yu Lin, Yiming Wang, Runyuan Cai +2 more · 2026

The paper introduces EDGE0, a streaming inference framework designed to serve large mixture-of-experts (MoE) models (such as 35B and 8B variants) on consumer devices constrained by limited RAM. It overcomes SSD loading latency by using a cross-token prerouter that predicts routing one token ahead, executes the prediction directly as the actual routing to completely overlap SSD expert streaming with compute, and restores accuracy loss from int4 quantization and routing replacement via an unmerged distillation LoRA.

Related Ideas

Is this a good idea?

Share onXLinkedIn