Hierarchical Dual-System Architecture for Asynchronous Strategic Deliberation in Real-Time Agents
Decouple real-time agent architectures into an asynchronous dual-cadence framework pairing a low-frequency large deliberative planner with a fast reactive policy.
Time
Compute level
Resources
What the paper used
Prerequisites
Scope
The Gap
In Section 3.1 and Section 10, Gamer relies on a single monolithic autoregressive policy (e.g., Qwen3-VL-8B or Qwen3.5-35B-A3B) to generate both executable 30 Hz actions and deliberative thinking tokens. Because real-time interaction imposes strict sub-100ms latency deadlines, thinking sequences must be kept extremely sparse (setting reasoning tokens to empty during routine steps), which leads to low success rates on complex logical puzzles like Abacus Circuitry (22.2%) and Hexanexus (33.3%).
“Third, although Gamer improves the trade-off between response latency and task performance in the evaluated settings, reasoning capability may remain a bottleneck in environments that demand deeper deliberation.”
Limitation
A single model cannot concurrently sustain deep multi-step planning and sub-100ms real-time control, causing severe performance bottlenecks on tasks requiring combinatorial problem-solving and long-range tactical planning.
Proposed Solution
Decouple the architecture into an asynchronous dual-cadence framework: a high-capacity deliberative 'System 2' planner (e.g., 72B parameter VLM) running asynchronously at low frequency (0.2-1 Hz) to generate topological waypoints, puzzle sub-goals, and inventory plans; and a fast 'System 1' reactive policy (e.g., 2B distilled VLA) operating synchronously at 30 Hz to execute keyboard-mouse actions conditioned on the latest System 2 latent goal representation.
The paper’s authors list this direction as future work, so they may already be pursuing it.
Why This Matters
Enables the agent to tackle complex strategic problems and puzzles without stalling real-time execution or suffering from control staleness in dynamic game environments.
How to Get Started
How to test it: Test whether decoupling a 72B VLM deliberative planner (0.2-1 Hz) from a 2B VLA reactive policy (30 Hz) improves complex problem-solving under real-time constraints by comparing against the monolithic Qwen3-VL-8B Gamer-RL baseline on the Bestial Ferocity side quest and the 40-task HSR minigame suite, measuring task/puzzle accuracy and end-to-end control latency. Stop if the dual-system agent fails to exceed Gamer-RL's 11.1% Bestial Ferocity puzzle accuracy and 49.7% minigame macro-average, or if System 1 action dispatch exceeds the 200 ms observation interval.
- Deploy a multi-GPU cluster hosting a high-capacity 72B parameter VLM for asynchronous System 2 planning.
- Fine-tune a 2B distilled VLA as a System 1 reactive policy to operate synchronously at 30 Hz.
- Set up game test environments including logical puzzles like Abacus Circuitry and Hexanexus.
- Benchmark the asynchronous dual-system architecture against monolithic baselines measuring task success rates and control latency.
Code linked to this paper on Hugging Face: github.com/RainBowLuoCS/Gamer
Suggested by AI. Adapt it to your setup.
Has This Been Done?
Harmonizing Real-Time Constraints and Long-Horizon Reasoning: An Asynchronous Agentic Framework for Dynamic Scheduling and HiBerNAC: Hierarchical brain-inspired robotic neural agent collective for disentangling complex manipulation explore asynchronous planning and dual frameworks for real-time constraints, but they do not specifically propose the combined 72B/2B VLM dual-cadence architecture operating at 0.2-1 Hz versus 30 Hz as detailed here.
- Harmonizing Real-Time Constraints and Long-Horizon Reasoning: An Asynchronous Agentic Framework for Dynamic Scheduling · 2026 · 0 citations
- HiBerNAC: Hierarchical brain-inspired robotic neural agent collective for disentangling complex manipulation · 2026 · 4 citations
We searched OpenAlex and arXiv for “Decouple real-time agent architectures into an asynchronous dual-cadence framework pairing a low-frequency large deliberative planner with a fast reactive policy.” and “Hierarchical Dual-System Architecture for Asynchronous Strategic Deliberation in Real-Time Agents” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game AgentRun Luo, Kuan Zhang, Qi Deng +24 more · 2026
This paper presents Gamer, a generalist Vision-Language-Action (VLA) foundation model designed for real-time, long-horizon interactive 3D and 2D gaming environments using raw screen observations and standard keyboard-mouse actions. The system employs a three-stage training pipeline comprising continued pre-training (CPT) with trajectory debiasing, multi-level supervised fine-tuning (SFT) with sparse reasoning and memory compaction, and Temporal-Aware Group Relative Policy Optimization (TA-GRPO) using game-native verifiers. To achieve sub-100ms latency necessary for real-time closed-loop control and rollout generation, the authors develop Streaming-vLLM, featuring stateful KV-cache reuse, incremental RoPE cache updates, and FSM-guided speculative decoding.
Related Ideas
- Reinforcement Learning-Driven Memory Lifecycle PolicyMemOS: A Memory OS for AI System🟠Medium
- Optimistic Synchronization for Cyclic DependenciesVery Large-Scale Multi-Agent Simulation in AgentScope🔴Hard
- Asynchronous RL for Multi-Turn Agentic TasksAReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning🟡Fairly easy
- Graph-Augmented Episodic Memory for Constraint-Aware Agentic PlanningEnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings🔴Hard
- Deadline-Budgeted Embodied Reasoning via Non-Verbal Spatial TokensEnvisioning Agentic Robotics: A New Era of Robotic Systems🟠Medium