Domain-Grounded Test-Time Compute Routing for Policy Workflows
Implement a domain-grounded test-time compute router that injects procedural RAG rules into the reasoning phase to satisfy strict constraint-heavy domains.
Time
Compute level
Resources
What the paper used
Prerequisites
Scope
The Gap
Figure 5 and Section 4.3 show that while increasing the 'thinking budget' (test-time compute) improves performance in some domains, it plateaus early in constraint-heavy domains like ITSM (peaking at 6.1%). Scaling reasoning tokens alone is insufficient.
“Increasing test-time compute yields substantial gains in most domains, but scaling is not universally monotonic: some domains plateau early, suggesting that additional reasoning tokens cannot compensate for fundamental gaps in domain knowledge or policy understanding.”
Limitation
Current scaling of test-time compute generates generic reasoning tokens that cannot compensate for fundamental gaps in understanding complex, domain-specific procedural rules and entity schemas.
Proposed Solution
Implement an adaptive, domain-grounded test-time compute router. Instead of simply generating more CoT tokens, inject targeted retrieval-augmented generation (RAG) queries that pull specific ITIL/HR procedural rules into the context window during the 'thinking' phase, dynamically allocating compute to verify specific SQL-like integrity constraints before action generation.
The paper’s authors list this direction as future work, so they may already be pursuing it.
Why This Matters
This approach optimizes inference costs by allocating test-time compute only where complex policy verification is needed, breaking through the performance ceilings observed in highly regulated enterprise domains.
How to Get Started
How to test it: Hypothesis: Routing test-time compute to targeted RAG queries over ITIL/HR procedural rules and schema constraints during thinking will overcome early domain plateaus better than generic reasoning tokens. Compare against GPT-OSS-120B (Medium and High thinking budgets) and ReAct baselines on ITSM, HR, and Email using pass@1 and Policy Compliance verifier scores across 3 runs; stop if the router fails to break the ITSM plateau (6.1% on GPT-OSS-120B) or fails to outperform High thinking budget at matched token cost.
- Construct a domain-specific vector database containing ITIL and HR procedure manuals.
- Implement a classification router that identifies constraint-heavy prompts requiring external rule verification.
- Modify the inference loop to dynamically query the RAG database and append rules to the chain-of-thought context window.
- Benchmark the updated system on ITSM evaluation tasks to measure improvements over naive CoT token scaling.
Code linked to this paper on Hugging Face: github.com/ServiceNow/EnterpriseOps-Gym
Suggested by AI. Adapt it to your setup.
Has This Been Done?
Papers like Towards Enforcing Company Policy Adherence in Agentic Workflows and Latency and Token-Aware Test-Time Compute explore workflow policy compliance and test-time compute allocation, but they leave open the specific mechanism of injecting targeted RAG rules during the thinking phase to verify procedural integrity constraints.
- MetaRoute-Bench: Evaluating Meta-Decision Policies for Agentic Workflow Routing · 2026 · 0 citations
- Latency and Token-Aware Test-Time Compute · 2025 · 8 citations
- Towards Enforcing Company Policy Adherence in Agentic Workflows · 2025 · 28 citations
We searched OpenAlex and arXiv for “Implement a domain-grounded test-time compute router that injects procedural RAG rules into the reasoning phase to satisfy strict constraint-heavy domains.” and “Domain-Grounded Test-Time Compute Routing for Policy Workflows” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise SettingsShiva Krishna Reddy Malay, Shravan Nayak, Jishnu Sethumadhavan Nair +6 more · 2026
This paper introduces ENTERPRISEOPS-GYM, a highly complex, sandboxed benchmark featuring 164 database tables and 512 tools designed to evaluate LLM agents in realistic, policy-constrained enterprise workflows. The study reveals that current frontier models struggle significantly with long-horizon strategic planning, cascading state dependencies, and safely refusing infeasible tasks, indicating they are not yet ready for autonomous enterprise deployment.
Related Ideas
- Retrieval-Augmented Policy Context for ScalabilityArch-Router: Aligning LLM Routing with Human Preferences🟡Fairly easy
- Process-Supervised Reasoning-Targeted Distillation for Routing-Replaced MoEsThe Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction🟠Medium
- Recursive Associative-Memory Graph Traversal for Multi-Step Scientific ReasoningWeight-Space Programming Doesn't Need Transformers🟡Fairly easy
- Requirement-Coverage and Execution-Trace-Aware Candidate Deliverable SelectionHarness^2: Recursive Agent Harnessing for an Open World🟢Easy
- Simulation-Based Ground Truth ValidationRemember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution🔴Hard