Hybrid Preference-Performance Routing Framework
Combine semantic matching probabilities with model capability predictions so query routing considers both text relevance and downstream model performance.
Source paper
Generated by AI. The paper’s authors did not write these.
Combine semantic matching probabilities with model capability predictions so query routing considers both text relevance and downstream model performance.
Quantify output uncertainty during policy routing to trigger clarification questions when incoming queries match overlapping or ambiguous policies.
Integrate dense retrieval before the router to dynamically inject only relevant policies into the prompt, removing hard limits on total policy count.
This paper introduces Arch-Router, a compact 1.5B parameter model designed to route LLM queries based on user-defined natural language policies rather than opaque performance benchmarks. By decoupling intent classification from model assignment and utilizing a synthetic data pipeline with Domain-Action taxonomies, the authors achieve state-of-the-art routing accuracy and reduced latency compared to proprietary models like GPT-4o.
The authors fine-tuned a Qwen-2.5-1.5B model on a synthetically generated dataset comprising 43k samples of multi-turn conversations labeled with Domain-Action routing policies. The model functions as a generative router that takes a user query and a list of XML-formatted policy descriptions as input, outputting the specific route identifier that best matches the user's intent.