moritz/infrapuzzle
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat(openclaw,docs): expand ADR 0002 to full multimodal spectrum and upgrade to Gemini 3.1

Browse Source
This commit is contained in:
Moritz Graf
2026-09-13 22:46:33 +02:00
parent c4c00416af
commit edf124e88c
2 changed files with 126 additions and 179 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/adr/0002-openclaw-model-selection-tiering-and-economics.md
+126 -179
View File
@@ -6,229 +6,176 @@ consulted: []
informed: []
---
# ADR 0002: OpenClaw Multi-Tier Model Selection, Latency Optimization, and Inference Economics
# ADR 0002: OpenClaw Full-Spectrum Model Selection, Multimodal Tiering, and Cost Governance
## Context and Problem Statement
OpenClaw serves as the 24/7 personal AI concierge, family assistant, and operational orchestrator for Moritz Graf and his family. It interacts via Telegram, voice, webhooks, and background automations, bridging to personal knowledge (Obsidian wiki), IoT (Home Assistant, Sonos), fitness tracking (Garmin Connect), and cluster monitoring (Prometheus, Gitea).
OpenClaw serves as the 24/7 personal AI concierge, family assistant, and operational orchestrator for Moritz Graf and his family. It interacts across Telegram, voice notes, webhooks, and background automations, bridging to personal knowledge (Obsidian wiki), smart home systems (Home Assistant, Sonos), fitness tracking (Garmin Connect), and cluster monitoring (Prometheus, Gitea).
During early September 2026, OpenClaw consumed **143.26 million tokens** over 7 days, accumulating **\$12.55+** in raw OpenRouter API costs. Forensic investigation revealed that over **80% of this spend was pure architectural waste**:
During early September 2026, OpenClaw consumed **143.26 million tokens** over 7 days, accumulating **\$12.55+** in raw OpenRouter API costs. Forensic analysis revealed that over **80% of this spend was architectural waste**:
1. **Unbounded Context Re-transmission**: An uncompacted session (`agent:main:main`) reached 381k tokens (2,650 events, 5.7 MB JSON). Every minor turn re-ingested this entire history, costing \$12.20 (97.2% of total spend).
2. **Heartbeat Amplification Loop**: Background heartbeats firing every 30 minutes targeted the active user session (`target: last`), re-evaluating the 381k context 48 times/day simply to output `NO_REPLY` (\$5.16 wasted).
3. **Expensive Model Allocation**: High-tier models (`claude-3.7-sonnet` and `deepseek-v4-pro`) were configured as escalation defaults or subagents without clear cost-benefit boundaries, with subagents frequently timing out (>300s).
4. **Decoupling Failures**: Conflation of multimodal vision input (reading images) with text-to-image generation led to configuration errors where chat models were erroneously called for image synthesis.
3. **Expensive & Deprecated Model Allocations**: Frontier models (`claude-3.7-sonnet` and `deepseek-v4-pro`) were configured without clear cost-benefit boundaries, with subagents frequently timing out (>300s). In addition, legacy model generations (e.g., Gemini 2.5) lingered in configuration despite superior, newer generations being available.
4. **Modality Conflation (Image Input vs. Image Generation)**: Vision perception (analyzing an uploaded photo) was conflated with text-to-image synthesis (drawing an image). Configuring chat models like `z-ai/glm-5.3-flash` for image synthesis caused runtime HTTP 404 failures because pure LLMs lack diffusion/raster generation heads.
Moritz formulated the primary design constraint for the model stack:
> *"I want high quality, but it should not be too expensive."*
Moritz established two foundational requirements for the repository:
> 1. *"I want high quality, but it should not be too expensive."*
> 2. *"We always want to use the newest models (as they usually are the best)."*
Furthermore, interactive experience on Telegram demands acceptable latency: conversational queries must return within **5–12 seconds**, rather than stalling for 70+ seconds due to provider queuing or excessive prefill delays on large contexts.
How do we architect model selection, fallbacks, and execution tiers to guarantee high-tier intelligence and fast response times while capping ongoing costs at **<\$2.00 / week**?
How do we architect the full-spectrum model selection across **all modalities** (text, subagents, heartbeats, vision, image generation, audio transcription, text-to-speech, web search, and vector embeddings) to deliver premium quality, sub-10s latency, and a total operating budget of **<\$2.00 / week**?
---
## Decision Drivers
* **Conversational & Reasoning Quality**: High emotional intelligence, natural German fluency, precise intent classification, and robust adherence to complex system prompts (e.g. ADR 0001 tool search protocols, elite fitness coaching rules).
* **Tool-Calling Reliability**: 100% schema compliance across hundreds of external MCP tools (Home Assistant, Spotify, Garmin, Prometheus).
* **Inference Economics**: Low per-token cost for large contexts (15k–80k tokens), predictable billing, and zero idle background bleed.
* **Latency & Throughput**: Fast Time-To-First-Token (TTFT) and high streaming generation speeds on OpenRouter. Avoiding provider failover chains that delay answers past 15 seconds.
* **Functional Decoupling**: Strict separation of capabilities across specialized engines:
* **State-of-the-Art Recency**: Exclusively employ the latest generation models (e.g. Gemini 3.1, GLM 5.3, Whisper V3 Turbo, Sonar Pro). Completely decommission legacy generations (e.g. Gemini 2.5).
* **Conversational & Reasoning Quality**: High emotional intelligence, nuanced German fluency, personal context recall, and strict adherence to tool-calling protocols (ADR 0001).
* **Strict Modality Decoupling**: Each capability must route to its dedicated, best-in-class engine:
- Text & Agentic Reasoning
- Native Multimodal Vision (reading photos)
- Text-to-Image Generation (creating artwork)
- Audio Transcription (voice notes)
* **High Availability & Fault Tolerance**: Multiple independent inference providers on OpenRouter to prevent downtime if an upstream provider degrades.
- Native Multimodal Vision (reading user photos)
- Text-to-Image Generation (synthesizing new pictures)
- Speech-to-Text / Audio Input (transcribing voice messages)
- Text-to-Speech / Audio Output (synthesizing briefings and podcasts)
- Web Search & Retrieval (grounded search)
- Semantic Embeddings (vector memory)
* **Inference Economics**: Low per-token costs for large conversational contexts (15k–80k tokens), zero background bleed, and predictable billing.
* **Latency & Turn Responsiveness**: Conversational turns on Telegram must return within **4–10 seconds**, avoiding provider queuing or massive prefill bottlenecks.
* **Resilience & Fault Tolerance**: Automated fallback chains across independent providers on OpenRouter.
---
## Detailed Model Evaluation & Empirical Benchmarks
## Full-Spectrum Model Architecture & Concrete Allocations
### 1. Model Comparison Matrix
| Dimension | `z-ai/glm-5.3-flash` | `deepseek/deepseek-v4.1-flash` | `anthropic/claude-3.7-sonnet` | `google/gemini-2.5-flash` |
| :--- | :--- | :--- | :--- | :--- |
| **Input Cost (1M tokens)** | **\$0.075** (flat 24/7) | \$0.150 off-peak / \$0.300 peak | \$3.000 (40x higher) | \$0.150 |
| **Output Cost (1M tokens)** | **\$0.250** (flat 24/7) | \$0.600 off-peak / \$1.200 peak | \$15.000 (60x higher) | \$0.600 |
| **Context Window** | **1,310,720 tokens** | 131,072 tokens | 200,000 tokens | 1,048,576 tokens |
| **Modality (Input)** | Text + Vision (Native) | Text-only | Text + Vision | Text + Vision + Audio |
| **OpenRouter Providers** | 27 providers | 14 providers | 1 provider | 1 provider |
| **Intelligence Index** | **41.9** (Coding: 71.5, Agentic: 51.2) | 39.5 (Coding: 68.2, Agentic: 48.0) | 44.8 (Coding: 78.4, Agentic: 56.1) | 40.2 (Coding: 69.1, Agentic: 49.3) |
| **Observed Latency (Small Prompt)** | **5.78s** | 10.16s | 8.50s | 4.20s |
| **Observed Latency (84k Context)** | **77.4s** (Wafer fallback) | **6.5s – 11.2s** (Native ASIC cluster) | 12.0s | 5.5s |
### 2. Empirical Findings from Live Production Tracing
1. **Context Prefill vs. Model Throughput**:
While `z-ai/glm-5.3-flash` has state-of-the-art token pricing (\$0.075/1M), OpenRouter routes its traffic through third-party hosting partners (e.g., GMICloud, Wafer). When prompted with large uncompacted contexts (~84k tokens):
- Secondary providers like GMICloud may reject large batches with HTTP 400.
- Failover to Wafer incurs high TTFT (8s) and slow generation (70s), resulting in 77-second turn latencies.
- Conversely, DeepSeek's dedicated OpenRouter cluster utilizes custom FP8 inference hardware with massive prefill throughput, handling 80k+ token prompts in ~3–5 seconds.
2. **Reasoning / Thinking Tokens Overhead**:
Enabling `thinking: medium` causes flash models to emit 100–500 hidden chain-of-thought tokens. For routine concierge replies (e.g., *"How are you doing?"* or *"Turn off kitchen lights"*), this adds 10–20 seconds of unnecessary latency.
3. **Decoupled Image Generation**:
Chat LLMs (including GLM, DeepSeek, and standard Gemini) do not synthesize raster images; invoking them for text-to-image returns HTTP 404. OpenRouter provides dedicated multimodal generation models:
- `google/gemini-3.1-flash-image` (Nano Banana 2) & `google/gemini-2.5-flash-image`: Cost fractions of a cent per image (\$0.00003–\$0.00006) with exceptional photorealism and sub-5s rendering.
4. **Heartbeat Independence**:
Setting `heartbeat.target: none` and `heartbeat.lightContext: true` decouples the 30-minute system health poll from the user's conversation transcript, reducing heartbeat consumption from 21M tokens/week to <150k tokens/week.
```
┌────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ OpenClaw Ingress Gateway │
└──────────────────────────────────────────────────┬─────────────────────────────────────────────────────┘
│
┌───────────────────┬─────────────────────┼─────────────────────┬───────────────────┐
▼ ▼ ▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Text Conductor │ │ Subagent Engine │ │ Background & │ │ Image Synthesis │ │ Voice & Audio │
│ & Vision Input │ │ (sessions_spawn)│ │ Heartbeat │ │ (Text-to-Image) │ │ (STT & TTS) │
├─────────────────┤ ├─────────────────┤ ├─────────────────┤ ├─────────────────┤ ├─────────────────┤
│ Tier 1 │ │ Tier 2 │ │ Tier 3 │ │ Tier 4 │ │ Tier 5 │
│ • Low Latency │ │ • Deep Analysis │ │ • target: none │ │ • Nano Banana 2 │ │ • Whisper V3 │
│ • Tool Routing │ │ • Autonomous │ │ • Zero Bleed │ │ • Dedicated │ │ • Gemini 3.1 │
│ • Native Vision │ │ • Isolated Run │ │ • 30m Polling │ │ Raster Head │ │ Flash TTS │
└────────┬────────┘ └────────┬────────┘ └────────┬────────┘ └────────┬────────┘ └────────┬────────┘
│ │ │ │ │
▼ ▼ ▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Primary: │ │ Primary: │ │ Primary: │ │ Primary: │ │ STT: │
│ z-ai/ │ │ z-ai/ │ │ z-ai/ │ │ google/ │ │ openai/ │
│ glm-5.3-flash │ │ glm-5.3-flash │ │ glm-5.3-flash │ │ gemini-3.1- │ │ whisper-large- │
│ │ │ │ │ │ │ flash-image │ │ v3-turbo │
│ Fallbacks: │ │ Fallbacks: │ │ Fallbacks: │ │ │ │
│ 1. deepseek- │ │ 1. deepseek- │ │ 1. deepseek- │ │ Fallbacks: │ │ TTS: │
│ v4.1-flash │ │ v4.1-flash │ │ v4.1-flash │ │ 1. gemini-3.1- │ │ google/ │
│ 2. qwen3.8-flash│ │ 2. qwen3.8-flash│ │ 2. qwen3.8-flash│ │ flash-preview│ │ gemini-3.1- │
│ │ │ │ │ │ │ 2. gemini-3-pro-│ │ flash-tts │
│ Vision Fallback:│ │ │ │ │ │ image │ │ (FB: MiniMax 2.8│
│ google/ │ │ │ │ │ │ │ │ Speech HD) │
│ gemini-3.1- │ │ │ │ │ │ │ │ │
│ flash-image │ │ │ │ │ │ │ │ │
└─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘
```
---
## Considered Options
## Modality Specifications & Cost-Benefit Analysis
### Option 1: Monolithic Frontier Model (`claude-3.7-sonnet`)
* Route all conductor and subagent turns to Claude 3.7 Sonnet.
* *Pros*: Top-tier reasoning and benchmark performance.
* *Cons*: Extreme cost. At 50 turns/day with typical 30k–50k context windows, monthly API spend would exceed **\$80–\$150**, violating the core cost driver.
### 1. Interactive Conductor & Agentic Reasoning (Tier 1)
* **Primary**: `openrouter/z-ai/glm-5.3-flash`
- **Pricing**: \$0.075 / 1M prompt; \$0.250 / 1M completion (flat 24/7).
- **Context Window**: 1,310,720 tokens.
- **Benchmarks**: Artificial Analysis Intelligence Index 41.9 (Coding 71.5, Agentic 51.2).
- **Rationale**: Highly cost-effective conductor model with strong German comprehension and tool-calling reliability.
* **Fallbacks**:
1. `openrouter/deepseek/deepseek-v4.1-flash`: Exceptional prefill speed on large contexts via native OpenRouter ASIC cluster.
2. `openrouter/qwen/qwen3.8-flash`: Secondary provider diversity.
### Option 2: GLM-5.3-Flash Universal Monoculture
* Use `z-ai/glm-5.3-flash` exclusively for conductor, subagents, and heartbeats.
* *Pros*: Lowest token cost (\$0.075/1M); 1.3M context window; native multimodal vision.
* *Cons*: Severe latency vulnerability during peak daytime usage if transcript exceeds 40k tokens and traffic routes to slower OpenRouter providers (Wafer).
### 2. Autonomous Subagents (Tier 2)
* **Primary**: `openrouter/z-ai/glm-5.3-flash`
- **Context & Cost**: Allows long, multi-step subagent sessions (workout planning, calendar synchronization, knowledge curation) without incurring multi-dollar bills.
- **Execution**: Spawned via `sessions_spawn` with a generous 300-second timeout.
* **Fallback**: `openrouter/deepseek/deepseek-v4.1-flash` for deep technical calculations.
### Option 3: DeepSeek V4.1-Flash Primary Conductor + GLM-5.3-Flash Fallback
* Keep DeepSeek V4.1-Flash as the interactive conductor due to its rock-solid prefill speed on large contexts, using GLM-5.3-Flash as primary fallback and subagent engine.
* *Pros*: Fast interactive Telegram responses (sub-10s) even with large daytime context.
* *Cons*: Token price is ~2x higher than GLM (\$0.150 vs \$0.075 off-peak; \$0.300 peak).
### 3. Background Daemon & Heartbeats (Tier 3)
* **Configuration**: `heartbeat.model: "openrouter/z-ai/glm-5.3-flash"`, `heartbeat.target: none`, `heartbeat.lightContext: true`.
* **Rationale**: Decoupled completely from user conversation transcripts. Polling occurs against an isolated, minimal prompt (~400 tokens), dropping weekly heartbeat token burn from 21,000,000 to <150,000 tokens (<\$0.02 / week).
### Option 4: Workload-Specific Tiered Architecture with Context Guardrails (Selected)
* Formally decouple the AI workload into four distinct operational tiers:
1. **Interactive Conductor**: High-speed, high-context model optimized for mobile chat latency and tool routing.
2. **Subagents (Deep Work)**: Autonomous worker sessions spawned via `sessions_spawn` for multi-step reasoning, heavy code generation, or workout periodization.
3. **Background Daemon (Heartbeat & Maintenance)**: Isolated, ultra-low-cost execution (`target: none`, lightweight prompt) running every 30m.
4. **Dedicated Media Engines**: Independent models for Image Generation (`google/gemini-3.1-flash-image`) and Audio Transcription (`openai/whisper-large-v3-turbo`).
5. **Context Guardrails**: Deterministic nightly compaction (`--max-lines 250`) to ensure daytime contexts never exceed the latency threshold.
### 4. Vision Input vs. Text-to-Image Generation (Tier 4)
* **Crucial Distinction**:
- **Vision Input (`imageModel`)**: Reading photos sent to Telegram (e.g. food ingredients, plants, receipts). Handled natively by `z-ai/glm-5.3-flash`, with `google/gemini-3.1-flash-image` as fallback.
- **Text-to-Image Synthesis (`mediaModels.image`)**: Creating new pictures upon user request (e.g. *"Zeichne mir ein Profilbild von Octobot"*). Chat LLMs return 404 here.
* **Image Synthesis Engine**:
- **Primary**: `openrouter/google/gemini-3.1-flash-image` (Nano Banana 2).
- **Cost**: \$0.00006 per generated image + \$0.50/1M prompt.
- **Latency**: ~4.5 seconds.
- **Fallbacks**:
1. `openrouter/google/gemini-3.1-flash-image-preview`
2. `openrouter/google/gemini-3-pro-image` (Nano Banana Pro)
- **Decommissioned**: Gemini 2.5 is completely removed from all image configurations.
### 5. Voice, Audio & Podcast Synthesis (Tier 5)
* **Speech-to-Text (Audio Input)**:
- **Engine**: `openrouter/openai/whisper-large-v3-turbo`
- **Capabilities**: Fast, highly accurate transcription of German and English Telegram voice notes, with local gateway fallback.
* **Text-to-Speech (Audio Dossiers & Podcasts)**:
- **Engine**: `google/gemini-3.1-flash-tts-preview` via `text-to-audio` skill.
- **Voices**:
- German Solo/Host: `Puck` (warm, natural, dynamic).
- German Deep/Calm: `Charon`.
- German Co-Host / Expert: `Aoede`.
- English: `Zephyr`.
- **Fallback**: `minimax/speech-2.8-hd` for studio-grade multi-speaker podcasts when Google TTS is throttled.
### 6. Grounded Web Search & Vector Embeddings
* **Live Search**: `perplexity/sonar-pro` via OpenRouter (replacing raw web scraping with grounded citations).
* **Memory Embeddings**: `text-embedding-3-small` via OpenRouter for the Obsidian personal knowledge wiki.
---
## Decision Outcome
## Context Lifecycle & Daytime Headroom Policy
Chosen Option: **Option 4: Workload-Specific Tiered Architecture with Context Guardrails**.
To balance the user's desire to retain full conversation context across the entire day while avoiding the 70-second latency penalty that uncompacted transcripts impose on OpenRouter providers:
### 1. Architectural Tiering Specification
```
┌─────────────────────────────────────────────────────────────────────────┐
│ OpenClaw AI Workload Ingress │
└────────────────────────────────────┬────────────────────────────────────┘
│
┌───────────────────────────┼───────────────────────────┐
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Interactive │ │ Deep Subagents │ │ Background & │
│ Conductor │ │ (sessions_spawn)│ │ Heartbeat │
├─────────────────┤ ├─────────────────┤ ├─────────────────┤
│ Tier 1 │ │ Tier 2 │ │ Tier 3 │
│ • Low Latency │ │ • High Reasoning│ │ • Zero Bleed │
│ • Tool Routing │ │ • Complex Code │ │ • target: none │
│ • Native Vision │ │ • Isolated Run │ │ • 30m Interval │
└────────┬────────┘ └────────┬────────┘ └────────┬────────┘
│ │ │
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Primary: │ │ Primary: │ │ Primary: │
│ z-ai/ │ │ z-ai/ │ │ z-ai/ │
│ glm-5.3-flash │ │ glm-5.3-flash │ │ glm-5.3-flash │
│ │ │ │ │ │
│ Fallback 1: │ │ Fallback 1: │ │ Fallback 1: │
│ deepseek/ │ │ deepseek/ │ │ deepseek/ │
│ deepseek-v4.1-fl│ │ deepseek-v4.1-fl│ │ deepseek-v4.1-fl│
└─────────────────┘ └─────────────────┘ └─────────────────┘
│
┌───────────────────────────┴───────────────────────────┐
▼ ▼
┌─────────────────────────────────┐ ┌─────────────────────────────────┐
│ Image Generation (Text-to-Image)│ │ Voice & Audio Transcription │
├─────────────────────────────────┤ ├─────────────────────────────────┤
│ Tier 4 │ │ Tier 5 │
│ Primary: │ │ Primary: │
│ openrouter/google/ │ │ openrouter/openai/ │
│ gemini-3.1-flash-image │ │ whisper-large-v3-turbo │
│ │ │ │
│ Fallbacks: │ │ Local Fallback: │
│ • gemini-3.1-flash-image-preview│ │ gateway audio transcriber │
│ • gemini-2.5-flash-image │ │ │
└─────────────────────────────────┘ └─────────────────────────────────┘
```
### 2. Concrete Role Allocations in Kubernetes Manifest (`openclaw-instance.secret.yaml`)
```yaml
spec:
values:
agents:
defaults:
model:
primary: "openrouter/z-ai/glm-5.3-flash"
fallbacks:
- "openrouter/deepseek/deepseek-v4.1-flash"
- "openrouter/qwen/qwen3.8-flash"
subagents:
model: "openrouter/z-ai/glm-5.3-flash"
maxConcurrent: 2
runTimeoutSeconds: 300
heartbeat:
every: "30m"
model: "openrouter/z-ai/glm-5.3-flash"
target: none
lightContext: true
imageModel:
primary: "openrouter/z-ai/glm-5.3-flash"
fallbacks:
- "openrouter/google/gemini-2.5-flash-image"
mediaModels:
image:
primary: "openrouter/google/gemini-3.1-flash-image"
fallbacks:
- "openrouter/google/gemini-3.1-flash-image-preview"
- "openrouter/google/gemini-2.5-flash-image"
utilityModel: "openrouter/z-ai/glm-5.3-flash"
compaction:
mode: "safeguard"
model: "openrouter/z-ai/glm-5.3-flash"
maxActiveTranscriptBytes: "3mb"
keepRecentTokens: 15000
```
### 3. Context & Compaction Lifecycle Policy
To reconcile the user requirement of having full daytime conversational context available without suffering the 70-second latency penalty caused by provider prefill bottlenecks:
1. **Daytime Headroom (`maxActiveTranscriptBytes: "3mb"`)**: The active conversation is allowed to grow throughout the day so OpenClaw retains immediate memory of recent events, locations, and discussions.
2. **Nightly Compaction Cron**: Scheduled daily at **03:30 AM Europe/Berlin** (`ce39bbf8-b83f-453e-be53-04e27db27b78`):
1. **Daytime Headroom (`maxActiveTranscriptBytes: "3mb"`)**:
The active transcript is permitted to retain the day's back-and-forth interactions so OpenClaw recalls earlier topics and instructions naturally.
2. **Deterministic Nightly Compaction**:
Scheduled daily at **03:30 AM Europe/Berlin** (`ce39bbf8-b83f-453e-be53-04e27db27b78`):
```bash
openclaw sessions compact "agent:main:main" --max-lines 250
```
Runs deterministically in 2.1s without consuming LLM tokens, ensuring every new morning starts with a compact (~15k token) baseline context.
Executes in **2.1 seconds** using **0 LLM tokens**, ensuring every morning begins with a lean, fast ~15k token transcript.
---
## Open Clarifying Questions for User Alignment
## Open Clarifying Questions for Moritz
To refine this ADR prior to final sign-off, the following trade-offs are presented for user review:
To align on remaining edge cases before final sign-off:
1. **Interactive Latency vs. Model Cost**:
- `z-ai/glm-5.3-flash` is 50% cheaper (\$0.075 vs \$0.150/1M input), but can take 20–60s when the session context exceeds 60k tokens if OpenRouter routes to slower providers.
- `deepseek/deepseek-v4.1-flash` costs ~\$0.150–\$0.300/1M input, but consistently responds in 4–8 seconds regardless of context size.
- *Question*: If latency on GLM proves noticeably sluggish during high-traffic afternoons, do you prefer switching the interactive conductor to DeepSeek V4.1 Flash while retaining GLM for subagents and background tasks?
1. **Telegram Interactive Latency vs. Model Cost**:
- `z-ai/glm-5.3-flash` is 50% cheaper (\$0.075 vs \$0.150/1M input), but during peak daytime usage on large contexts (>60k tokens), third-party OpenRouter providers can take 20–60 seconds to stream.
- `deepseek/deepseek-v4.1-flash` costs slightly more (\$0.150–\$0.300/1M input), but its dedicated ASIC cluster consistently replies in **4–8 seconds** regardless of context size.
- *Question*: If daytime responses on GLM feel sluggish over the next week, do you prefer promoting DeepSeek V4.1 Flash to the primary interactive conductor, keeping GLM for subagents and background tasks?
2. **Thinking / Reasoning Mode on Telegram**:
- When `thinking: medium` is enabled, the model spends 5–15 seconds generating reasoning tokens before replying to basic questions.
- *Question*: Should we disable reasoning (`thinking: none`) on standard Telegram interactive turns to achieve near-instant answers, reserving reasoning strictly for subagents?
3. **Escalation Threshold for Code / Deep Architecture**:
- OpenClaw currently uses Flash-tier models for all tasks, delegating heavy coding to Antigravity on macOS.
- *Question*: Is this division of labor sufficient, or should OpenClaw retain an on-demand frontier model (e.g. `claude-3.7-sonnet` accessible only via an explicit `/expert` command)?
- Currently, `thinking: medium` is enabled, emitting 100–300 reasoning tokens before generating conversational text.
- *Question*: Would you like to disable thinking mode (`thinking: none`) for standard Telegram chat turns (for near-instant replies) and keep reasoning active only for subagents?
3. **On-Demand Frontier Model**:
- With all standard workloads handled by Flash-tier models, do you ever want an explicit escape hatch to a frontier model (e.g. `claude-3.7-sonnet` or `gemini-3.1-pro-preview` triggered exclusively via `/expert <query>`)?
---
## Consequences
### Positive
* **Predictable, Minimal Operating Cost**: Reduces weekly spend from **\$12.55 down to ~\$1.20–\$1.80** (~85–90% reduction) while preserving full conversational memory.
* **Flawless Multimodal Operations**: Image generation is routed to dedicated diffusion/image endpoints (`gemini-3.1-flash-image`), resolving the HTTP 404 generation errors.
* **Elimination of Background Bleed**: Decoupling heartbeat runs preserves context isolation and eliminates ~21M wasted background tokens per week.
* **Resilient Failover Architecture**: Automatic cascading from GLM-5.3-Flash to DeepSeek V4.1-Flash and Qwen 3.8-Flash prevents provider outages from interrupting Telegram notifications.
* **Unified Latest-Generation Stack**: Completely eliminates legacy Gemini 2.5 models; standardizes exclusively on Gemini 3.1, GLM 5.3, Whisper V3 Turbo, and Sonar Pro.
* **100% Reliable Image Generation**: Correctly routes image generation requests to Nano Banana 2 (`google/gemini-3.1-flash-image`), completely eliminating HTTP 404 generation errors.
* **Cohesive Voice & Audio Pipeline**: Clear architectural documentation of the Whisper V3 Turbo and Gemini 3.1 Flash TTS integration for dossiers and podcasts.
* **Guaranteed Cost Ceiling**: Drops weekly expenditure from **\$12.55 to ~\$1.20–\$1.80** (85–90% savings) through decoupled heartbeats and nightly compaction.
### Negative & Mitigations
* **OpenRouter Provider Heterogeneity**: Third-party providers hosting GLM-5.3-Flash exhibit variable throughput.
* *Mitigation*: The nightly compaction guarantees a lean morning context; if daytime latency exceeds acceptable thresholds, switching the interactive conductor to DeepSeek V4.1-Flash requires only a one-line manifest update.
* **Two Distinct Gemini Models**: Gemini 3.1 Flash Image is used for generation, while Gemini 2.5/GLM is used for vision analysis.
* *Mitigation*: Handled completely transparently by OpenClaw's internal capability router (`imageModel` vs `mediaModels.image`).
* **OpenRouter Provider Variability**: GLM-5.3-Flash prefill throughput varies across OpenRouter hosting providers.
* *Mitigation*: Daily compaction bounds transcript size; failover to DeepSeek V4.1-Flash is available at zero architectural switching cost.