Skip to content
AI-generated research idea🟡Fairly easy🔧Minor TweakRelated work found

Domain-Grounded Test-Time Compute Routing for Policy Workflows

Implement a domain-grounded test-time compute router that injects procedural RAG rules into the reasoning phase to satisfy strict constraint-heavy domains.

Time

3–5 weeks to index domain policy rules and table schemas, build the adaptive test-time compute router, and run 3-seed ablations against thinking-budget baselines.

Compute level

Large cluster

Resources

A 4–8x H100 80GB GPU cluster (or equivalent API inference budget) to serve GPT-OSS-120B across thinking-budget baselines and adaptive RAG-routed runs over 3 seeds on ~470 ITSM/HR/Email tasks, plus vector-store indexing and Dockerized sandbox containers.

What the paper used

Hardware not stated in the paper; evaluates 14 models (closed- and open-source up to 235B parameters) over 1,150 tasks across 3 runs each in a Docker sandbox (164 database tables, 512 tools) at reported inference costs of $0.007–$0.36 per task.

Prerequisites

Prompt engineering, retrieval-augmented generation, inference scaling

Scope

Modify existing code and agent system prompts

The Gap

Figure 5 and Section 4.3 show that while increasing the 'thinking budget' (test-time compute) improves performance in some domains, it plateaus early in constraint-heavy domains like ITSM (peaking at 6.1%). Scaling reasoning tokens alone is insufficient.

“Increasing test-time compute yields substantial gains in most domains, but scaling is not universally monotonic: some domains plateau early, suggesting that additional reasoning tokens cannot compensate for fundamental gaps in domain knowledge or policy understanding.”
From the paper, Section 5, page 10. Matched against the PDF text.

Limitation

Current scaling of test-time compute generates generic reasoning tokens that cannot compensate for fundamental gaps in understanding complex, domain-specific procedural rules and entity schemas.

Proposed Solution

Implement an adaptive, domain-grounded test-time compute router. Instead of simply generating more CoT tokens, inject targeted retrieval-augmented generation (RAG) queries that pull specific ITIL/HR procedural rules into the context window during the 'thinking' phase, dynamically allocating compute to verify specific SQL-like integrity constraints before action generation.

The paper’s authors list this direction as future work, so they may already be pursuing it.

Why This Matters

This approach optimizes inference costs by allocating test-time compute only where complex policy verification is needed, breaking through the performance ceilings observed in highly regulated enterprise domains.

How to Get Started

How to test it: Hypothesis: Routing test-time compute to targeted RAG queries over ITIL/HR procedural rules and schema constraints during thinking will overcome early domain plateaus better than generic reasoning tokens. Compare against GPT-OSS-120B (Medium and High thinking budgets) and ReAct baselines on ITSM, HR, and Email using pass@1 and Policy Compliance verifier scores across 3 runs; stop if the router fails to break the ITSM plateau (6.1% on GPT-OSS-120B) or fails to outperform High thinking budget at matched token cost.

  1. Construct a domain-specific vector database containing ITIL and HR procedure manuals.
  2. Implement a classification router that identifies constraint-heavy prompts requiring external rule verification.
  3. Modify the inference loop to dynamically query the RAG database and append rules to the chain-of-thought context window.
  4. Benchmark the updated system on ITSM evaluation tasks to measure improvements over naive CoT token scaling.

Code linked to this paper on Hugging Face: github.com/ServiceNow/EnterpriseOps-Gym

Suggested by AI. Adapt it to your setup.

Has This Been Done?

Related work found

Papers like Towards Enforcing Company Policy Adherence in Agentic Workflows and Latency and Token-Aware Test-Time Compute explore workflow policy compliance and test-time compute allocation, but they leave open the specific mechanism of injecting targeted RAG rules during the thinking phase to verify procedural integrity constraints.

We searched OpenAlex and arXiv for “Implement a domain-grounded test-time compute router that injects procedural RAG rules into the reasoning phase to satisfy strict constraint-heavy domains.” and “Domain-Grounded Test-Time Compute Routing for Policy Workflows” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings

Shiva Krishna Reddy Malay, Shravan Nayak, Jishnu Sethumadhavan Nair +6 more · 2026

This paper introduces ENTERPRISEOPS-GYM, a highly complex, sandboxed benchmark featuring 164 database tables and 512 tools designed to evaluate LLM agents in realistic, policy-constrained enterprise workflows. The study reveals that current frontier models struggle significantly with long-horizon strategic planning, cascading state dependencies, and safely refusing infeasible tasks, indicating they are not yet ready for autonomous enterprise deployment.

Related Ideas

Is this a good idea?

Share onXLinkedIn