diff --git a/k8s/openclaw/TROUBLESHOOTING.md b/k8s/openclaw/TROUBLESHOOTING.md index cdd304b..60e612b 100644 --- a/k8s/openclaw/TROUBLESHOOTING.md +++ b/k8s/openclaw/TROUBLESHOOTING.md @@ -906,4 +906,34 @@ This document captures fixed issues, operational quirks, and incident root-cause - For agents that routinely process user photos or document scans, assign dedicated vision models (`google/gemini-3.8-flash`) as the agent's primary model rather than relying on `agents.defaults.imageModel`. - Maintain the response guarantee in agent prompts to prevent empty turns when reasoning models encounter uncertainty. +--- + +### Issue 28: Topic 20 #Sport Transcript Blindness: GLM-5.3-Flash Reasoning Hallucination on Audio Inbound Attachments + +- **Date**: 2026-10-12 +- **Severity**: Medium (User sent 24s voice note requesting Cable Workout A evaluation from Garmin; transcript was echoed to Telegram but agent claimed no transcript arrived) +- **Symptom**: + 1. The user recorded a 24s voice note asking for a Garmin evaluation of "Cable Workout A" (history, frequency, progression). + 2. The gateway transcribed the audio correctly via Whisper and echoed it back to Telegram (`messageId: 2739`). + 3. The prompt delivered to the agent contained the full transcript: `[Audio]\nTranscript:\nIch habe gerade ein Workout gemacht, das Cable Workout A...`. + 4. Yet after 78 tool steps and 19.5 minutes of looping, the agent (`fitness`) replied: + *"Moin mop 👋 Dein Sprachnachricht ist bei mir ohne Transkript angekommen – ich kann nur raten, was drin war. 🎙️"*. +- **Root Cause**: + 1. **GLM-5.3-Flash Multimodal Confusion**: + The message context included both the text transcript and the staged `.ogg` attachment in `message.media`. + In its internal reasoning (`thinking`), GLM-5.3-Flash hallucinated: + *"Voice message from Moritz. Need transcript — it's not provided this time? The context shows a voice message but no transcript. Hmm, the inbound is voice; transcript usually would be in context. Not visible. Maybe I need to transcribe."* + The model fixated on the audio file attachment, concluded it had to transcribe the file itself, and completely ignored the actual transcript string right below `[Audio]\nTranscript:`. + 2. **19.5-Minute Looping Failure**: + The agent attempted to locate local audio tools (`which whisper ffmpeg`), searched memory files, found an unrelated October 7 note about protein powder in the wiki, and spent 78 turns cycling between file reads until giving up with a false negative. +- **Resolution**: + 1. **Model Upgraded to Gemini 3.8 Flash**: + Replaced `primary: "openrouter/z-ai/glm-5.3-flash"` with `openrouter/google/gemini-3.8-flash` (fallbacks: `glm-5.3-flash`, `deepseek-v4.1-flash`, `qwen3.8-flash`) for agent `fitness` both live via `openclaw config set` and declaratively in `k8s/openclaw/openclaw-instance.secret.yaml`. Gemini 3.8 Flash natively handles multimodal metadata without overlooking inline text transcripts. + 2. **Garmin Workout Data Synced & Evaluated**: + Executed `garmin_sync.py` to mirror the latest workouts into `data/garmin/activities/2026-10.jsonl`. + Retrieved all 3 sessions of Moritz's `SuperSetACable-55mins` (2026-09-28, 2026-10-06, 2026-10-12) and extracted the workout progression data. +- **How to Avoid Recurrence**: + - Keep interactive topic agents receiving media or voice notes on Gemini 3.8 Flash or DeepSeek V4.1 Flash rather than GLM-5.3-Flash. + - In agent system prompts, reinforce that `[Audio]\nTranscript:\n` already contains the authoritative, finalized speech-to-text result and requires no local audio file re-processing. + diff --git a/k8s/openclaw/openclaw-instance.secret.yaml b/k8s/openclaw/openclaw-instance.secret.yaml index cb39e93..e19f005 100644 Binary files a/k8s/openclaw/openclaw-instance.secret.yaml and b/k8s/openclaw/openclaw-instance.secret.yaml differ