fix(openclaw): switch recherche agent to deepseek-v4.1-flash and document STT quota failure
This commit is contained in:
@@ -98,9 +98,11 @@ How do we architect the full-spectrum model selection across **all modalities**
|
||||
2. `openrouter/qwen/qwen3.8-flash`: Provider diversity.
|
||||
|
||||
### 2. Autonomous Subagents (Tier 2)
|
||||
* **Primary (Active Trial)**: `openrouter/z-ai/glm-5.3-flash`
|
||||
* **General Subagents**: `openrouter/z-ai/glm-5.3-flash`
|
||||
- **Execution**: Spawned via `sessions_spawn` with a 300-second timeout. Inherits the `tool_call` protocol guardrails from `AGENTS.md`.
|
||||
* **Fallback**: `openrouter/deepseek/deepseek-v4.1-flash` for high-concurrency tool execution fallback.
|
||||
* **Deep Analysis & IT Specialists (`itguy`, `recherche`)**: `openrouter/deepseek/deepseek-v4.1-flash`
|
||||
- High-precision reasoning, robust nested tool loops without syntax drift, and immune to XML leakage on extensive research queries or CLI diagnostics.
|
||||
* **Fallback**: `openrouter/deepseek/deepseek-v4.1-flash` / `openrouter/qwen/qwen3.8-flash`.
|
||||
|
||||
### 3. Background Daemon & Heartbeats (Tier 3)
|
||||
* **Configuration**: `heartbeat.model: "openrouter/z-ai/glm-5.3-flash"`, `heartbeat.target: none`, `heartbeat.lightContext: true`.
|
||||
|
||||
@@ -19,6 +19,7 @@ This document captures fixed issues, operational quirks, and incident root-cause
|
||||
| **2026-09-28** | **Token Saturation & Model Degradation** | Subagents emitted hallucinatory slang/jargon (e.g. Hebamme with `Sleep-Hax0r`) and Node.js event loop delayed 31s under 2GB RSS | `RESOLVED` | Identified uncompacted 127k-token sessions causing reasoning collapse in GLM-5.3-Flash. Compacted live bloated sessions and automated multi-agent nightly compaction via `setup-compaction-cron.sh`. |
|
||||
| **2026-09-29** | **Hardlink Regression & Disk I/O** | Pod crashed with SIGKILL (137), bot unresponsive, startup stuck in `Dl` state opening agent SQLite DBs | `RESOLVED` | Replaced recurring `ln -f` in `cronjob-wiki-backup.secret.yaml` with `cp --remove-destination`, decoupled live workspace files, warmed Linux page cache for 12 agent DBs on HDD storage, restoring `5/5 Running`. |
|
||||
| **2026-09-29** | **Compaction Pitfall & Split-Turn Drift** | Shopping agent ignored user message (Küchenwaage) and replied with old supermarket wiki edit in broken German | `RESOLVED` | Identified `--max-lines 250` bypassing LLM compaction on sessions with <250 lines but 40k-95k tokens. Switched to true LLM summarization, compacted live sessions, added event-loop pacing, and documented split-turn context recovery. |
|
||||
| **2026-09-29** | **Audio & Speech-to-Text** | Telegram voice notes failed transcription with `reason=ProviderHttpError`, passing empty text (`0 chars`) and inducing bot hallucinations | `RESOLVED` | Identified OpenRouter audio balance floor ($0.50) causing HTTP 402 when key quota was within $0.50 of monthly limit. Topped up buffer, verified transcription, and switched Recherche agent to DeepSeek V4.1 Flash. |
|
||||
|
||||
---
|
||||
|
||||
@@ -248,4 +249,44 @@ This document captures fixed issues, operational quirks, and incident root-cause
|
||||
- Monitor session token sizes (`openclaw sessions --all-agents --json`) rather than line counts.
|
||||
- Keep `setup-compaction-cron.sh` aligned across git and live PVCs.
|
||||
|
||||
---
|
||||
|
||||
### Issue 9: Speech-to-Text / Audio Transcription Failure (`ProviderHttpError`) & Empty Voice Note Transcripts
|
||||
|
||||
- **Date**: 2026-09-29
|
||||
- **Severity**: High (Telegram voice notes fail transcription, passing empty messages to agents and prompting hallucinated/frantic behavior)
|
||||
- **Symptom**:
|
||||
1. Telegram voice notes fail transcription with:
|
||||
```text
|
||||
[media-understanding] audio: failed (0/1) reason=ProviderHttpError
|
||||
```
|
||||
in ~300–500ms.
|
||||
2. The inbound Telegram message is delivered to the agent with `0 chars` text (`(group, audio/ogg, 0 chars)`).
|
||||
3. The agent receives an empty prompt or placeholder (`[Audio attachment could not be analyzed]`). It does not know what the user requested, resulting in:
|
||||
- No reply,
|
||||
- Completely hallucinated or unrelated replies, or
|
||||
- Frantic autonomous attempts by the agent to find and transcribe the audio file in `/home/node/.openclaw/media/inbound/` via terminal commands (`ffmpeg`, `python3`, `uv`, free model APIs), producing errors and strange messages that give the appearance the bot was "hacked".
|
||||
- **Root Cause**:
|
||||
1. **OpenRouter Audio Balance Floor ($0.50)**:
|
||||
OpenRouter requires at least **$0.50 in credit buffer / remaining key limit** for any audio transcription request (`openai/whisper-large-v3-turbo` or multimodal audio). When the monthly limit was set to $70.00 and usage reached ~$59.80 (leaving <$0.50 of head room under the prior limit), OpenRouter rejected audio requests with HTTP 402:
|
||||
```json
|
||||
{"error":{"message":"This request requires at least $0.50 in balance for audio","code":402,"metadata":{"limit_source":"openrouter_key_limit"}}}
|
||||
```
|
||||
2. **Chat Completions Succeeded while Audio Failed**:
|
||||
Because text completions cost only fractions of a cent ($0.0001–$0.001), regular chat queries continued to work normally. Only audio transcriptions were blocked by the $0.50 threshold.
|
||||
3. **Error Truncation in OpenClaw**:
|
||||
In `runner.entries-LWAceiTW.js`, the error was processed by `summarizeDecisionReason(reason)`, which executes `normalizeDecisionReason(reason).split(":")[0]`. Because `ProviderHttpError` was not stripped by the regex, the error string `ProviderHttpError: OpenRouter audio transcription failed (HTTP 402)...` was truncated to `ProviderHttpError`. This obscured the HTTP 402 status code and key limit message in normal logs.
|
||||
4. **Single Provider without Fallback**:
|
||||
`openclaw-instance.secret.yaml` only declared a single audio provider (`openrouter/openai/whisper-large-v3-turbo`) with no fallback provider configured. When OpenRouter returned 402, OpenClaw marked the audio as failed and fell back to empty input.
|
||||
- **Resolution**:
|
||||
1. **Buffer Headroom Verified**:
|
||||
Confirmed OpenRouter key monthly limit has ample remaining buffer ($10.19 remaining), and re-tested live audio transcription of the exact failed 52-second Telegram voice files (`b9e0da37...` and `aef0424f...`), confirming fast, accurate transcription (HTTP 200).
|
||||
2. **Recherche Agent Upgraded**:
|
||||
Switched `recherche` agent from `openrouter/z-ai/glm-5.3-flash` to `openrouter/deepseek/deepseek-v4.1-flash` in `openclaw-instance.secret.yaml` and live `openclaw.json` to ensure robust analytical reasoning and prevent prompt confusion.
|
||||
- **How to Avoid Recurrence**:
|
||||
- Maintain a sufficient credit buffer on OpenRouter (at least $5–$10 remaining monthly headroom).
|
||||
- Interpret `reason=ProviderHttpError` on audio in OpenClaw as an upstream OpenRouter HTTP 402 (balance/limit floor) or HTTP 429/502.
|
||||
- Review adding a secondary fallback STT provider if OpenRouter audio is unavailable.
|
||||
|
||||
|
||||
|
||||
|
||||
Binary file not shown.
Reference in New Issue
Block a user