Skip to content
AI-generated research idea🟠Medium💡Moderate InnovationRelated work found

Hierarchical Dual-System Architecture for Asynchronous Strategic Deliberation in Real-Time Agents

Decouple real-time agent architectures into an asynchronous dual-cadence framework pairing a low-frequency large deliberative planner with a fast reactive policy.

Time

6 to 10 weeks (2-3 weeks for 2B policy distillation and System 2 subgoal data curation/SFT, 2-3 weeks for asynchronous dual-cadence Streaming-vLLM integration, and 2-4 weeks for online RL and benchmark evaluation).

Compute level

Large cluster

Resources

A distributed H200/A800 training cluster (comparable to the paper's ~45K H200 GPU-hours) for 2B VLA distillation and 72B System 2 SFT/RL on ~2.4K CPT hours, ~1K SFT hours, and ~14.6K RL rollouts, plus 64-128 NVIDIA H200 GPUs in a sharded Streaming-vLLM pool serving 32 parallel Windows game VMs to co-host both models during online RL and evaluation.

What the paper used

Controlled CPT scaling used 120 configurations across Qwen3.5 Dense (0.8B, 2B, 9B, 27B) and ~45K NVIDIA H200 GPU-hours. The main Gamer model (Qwen3-VL-8B; also Qwen3.5-35B-A3B MoE in CPT) trained on ~2.4K hours gameplay plus ~6.3B VLM tokens on a distributed H200 cluster (CPT), ~1K hours HSR annotations (~706K samples) on a distributed A800 cluster (SFT), and ~14.6K online rollouts using Megatron/MCore with 64 NVIDIA H200 GPUs serving 32 parallel Windows VMs (RL).

Prerequisites

Multi-agent systems, asynchronous inference orchestration, vision-language-action architectures.

Scope

Build asynchronous communication channels between two model instances in the serving engine and retrain the reactive controller conditioned on planner tokens.

The Gap

In Section 3.1 and Section 10, Gamer relies on a single monolithic autoregressive policy (e.g., Qwen3-VL-8B or Qwen3.5-35B-A3B) to generate both executable 30 Hz actions and deliberative thinking tokens. Because real-time interaction imposes strict sub-100ms latency deadlines, thinking sequences must be kept extremely sparse (setting reasoning tokens to empty during routine steps), which leads to low success rates on complex logical puzzles like Abacus Circuitry (22.2%) and Hexanexus (33.3%).

“Third, although Gamer improves the trade-off between response latency and task performance in the evaluated settings, reasoning capability may remain a bottleneck in environments that demand deeper deliberation.”
From the paper, Section 10, page 28. Matched against the PDF text.

Limitation

A single model cannot concurrently sustain deep multi-step planning and sub-100ms real-time control, causing severe performance bottlenecks on tasks requiring combinatorial problem-solving and long-range tactical planning.

Proposed Solution

Decouple the architecture into an asynchronous dual-cadence framework: a high-capacity deliberative 'System 2' planner (e.g., 72B parameter VLM) running asynchronously at low frequency (0.2-1 Hz) to generate topological waypoints, puzzle sub-goals, and inventory plans; and a fast 'System 1' reactive policy (e.g., 2B distilled VLA) operating synchronously at 30 Hz to execute keyboard-mouse actions conditioned on the latest System 2 latent goal representation.

The paper’s authors list this direction as future work, so they may already be pursuing it.

Why This Matters

Enables the agent to tackle complex strategic problems and puzzles without stalling real-time execution or suffering from control staleness in dynamic game environments.

How to Get Started

How to test it: Test whether decoupling a 72B VLM deliberative planner (0.2-1 Hz) from a 2B VLA reactive policy (30 Hz) improves complex problem-solving under real-time constraints by comparing against the monolithic Qwen3-VL-8B Gamer-RL baseline on the Bestial Ferocity side quest and the 40-task HSR minigame suite, measuring task/puzzle accuracy and end-to-end control latency. Stop if the dual-system agent fails to exceed Gamer-RL's 11.1% Bestial Ferocity puzzle accuracy and 49.7% minigame macro-average, or if System 1 action dispatch exceeds the 200 ms observation interval.

  1. Deploy a multi-GPU cluster hosting a high-capacity 72B parameter VLM for asynchronous System 2 planning.
  2. Fine-tune a 2B distilled VLA as a System 1 reactive policy to operate synchronously at 30 Hz.
  3. Set up game test environments including logical puzzles like Abacus Circuitry and Hexanexus.
  4. Benchmark the asynchronous dual-system architecture against monolithic baselines measuring task success rates and control latency.

Code linked to this paper on Hugging Face: github.com/RainBowLuoCS/Gamer

Suggested by AI. Adapt it to your setup.

Has This Been Done?

Related work found

Harmonizing Real-Time Constraints and Long-Horizon Reasoning: An Asynchronous Agentic Framework for Dynamic Scheduling and HiBerNAC: Hierarchical brain-inspired robotic neural agent collective for disentangling complex manipulation explore asynchronous planning and dual frameworks for real-time constraints, but they do not specifically propose the combined 72B/2B VLM dual-cadence architecture operating at 0.2-1 Hz versus 30 Hz as detailed here.

We searched OpenAlex and arXiv for “Decouple real-time agent architectures into an asynchronous dual-cadence framework pairing a low-frequency large deliberative planner with a fast reactive policy.” and “Hierarchical Dual-System Architecture for Asynchronous Strategic Deliberation in Real-Time Agents” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game Agent

Run Luo, Kuan Zhang, Qi Deng +24 more · 2026

This paper presents Gamer, a generalist Vision-Language-Action (VLA) foundation model designed for real-time, long-horizon interactive 3D and 2D gaming environments using raw screen observations and standard keyboard-mouse actions. The system employs a three-stage training pipeline comprising continued pre-training (CPT) with trajectory debiasing, multi-level supervised fine-tuning (SFT) with sparse reasoning and memory compaction, and Temporal-Aware Group Relative Policy Optimization (TA-GRPO) using game-native verifiers. To achieve sub-100ms latency necessary for real-time closed-loop control and rollout generation, the authors develop Streaming-vLLM, featuring stateful KV-cache reuse, incremental RoPE cache updates, and FSM-guided speculative decoding.

Related Ideas

Is this a good idea?

Share onXLinkedIn