diff --git a/k8s/openclaw/TROUBLESHOOTING.md b/k8s/openclaw/TROUBLESHOOTING.md index 60e612b..d168207 100644 --- a/k8s/openclaw/TROUBLESHOOTING.md +++ b/k8s/openclaw/TROUBLESHOOTING.md @@ -933,7 +933,37 @@ This document captures fixed issues, operational quirks, and incident root-cause Executed `garmin_sync.py` to mirror the latest workouts into `data/garmin/activities/2026-10.jsonl`. Retrieved all 3 sessions of Moritz's `SuperSetACable-55mins` (2026-09-28, 2026-10-06, 2026-10-12) and extracted the workout progression data. - **How to Avoid Recurrence**: - - Keep interactive topic agents receiving media or voice notes on Gemini 3.8 Flash or DeepSeek V4.1 Flash rather than GLM-5.3-Flash. - In agent system prompts, reinforce that `[Audio]\nTranscript:\n` already contains the authoritative, finalized speech-to-text result and requires no local audio file re-processing. +### Issue 29: Decoupled Vision Perception — Gemini 3.8 Flash Media-Understanding with GLM-5.3-Flash Primary Chat Engine +- **Symptoms**: + - Switching the primary agent model completely to `google/gemini-3.8-flash` changed the entire conversational tone and agent persona across all text turns, whereas only high-fidelity OCR and nutrition image perception were desired. + - When `openrouter/z-ai/glm-5.3-flash` was the primary model, OpenClaw previously routed photos natively to GLM because the OpenRouter model catalog declared multimodal capability (`image`), causing GLM to fail at OCR reading of nutritional labels. +- **Root Cause & Architectural Discovery**: + 1. OpenClaw checks `activeModelSupportsNativeVision` and `resolveRunModelHasVision`. If the primary model advertises vision on OpenRouter AND no explicit image understanding models are declared in `tools.media.models`, OpenClaw bypasses the `media-understanding` pipeline and passes raw images natively to the chat engine. + 2. Because GLM-5.3-Flash struggled with German nutritional tables, the entire primary model had been switched to Gemini 3.8 Flash, which Moritz explicitly rejected ("Haben wir nun das Modell für den Koch agent komplett auf Gemini 3.8 flash gewechselt? Das wollte ich nicht. Nur für die Bilderkennung bitte."). +- **Resolution**: + 1. **Reverted Primary Chat Models**: + Reverted `agents.entries.koch.model.primary` and `agents.entries.fitness.model.primary` back to `openrouter/z-ai/glm-5.3-flash` (with fallbacks `deepseek-v4.1-flash`, `qwen3.8-flash`) in both `openclaw.json` and `openclaw-instance.secret.yaml`. + 2. **Configured Dedicated Media-Understanding Image Engine**: + Added an explicit `image` capability model under `tools.media.models`: + ```yaml + tools: + media: + image: + enabled: true + preferredModel: "openrouter/google/gemini-3.8-flash" + models: + - provider: openrouter + model: "google/gemini-3.8-flash" + capabilities: ["image"] + prompt: "Beschreibe das Bild detailliert auf Deutsch. Wenn Nährwertangaben, Zutaten, Barcode, Verpackungstext oder handschriftliche Notizen zu sehen sind, transkribiere alle Zahlen, Gramm-Angaben und Nährwerte (Kalorien, Eiweiß/Protein, Kohlenhydrate, Fett) vollständig und exakt." + ``` + 3. **Declared Text-Only Override for GLM**: + Under `models.providers.openrouter.models`, registered `z-ai/glm-5.3-flash` with `input: ["text"]`. This signals to OpenClaw that GLM must never receive raw images natively, forcing automatic routing of all image attachments through Gemini 3.8 Flash media-understanding before handing off enriched text to GLM. + 4. **Updated Persisted SQLite Sessions**: + Reset `session_nodes` for `agent:koch:telegram:group:-1004326921510:topic:21` and `agent:fitness:telegram:group:-1004326921510:topic:20` to `z-ai/glm-5.3-flash`. +- **How to Avoid Recurrence**: + - Always decouple perception tasks (OCR, image description, speech-to-text) into `tools.media` or `imageModel`, keeping primary conversational models focused on reasoning and dialogue without changing the core chat persona. + diff --git a/k8s/openclaw/openclaw-instance.secret.yaml b/k8s/openclaw/openclaw-instance.secret.yaml index e19f005..7401094 100644 Binary files a/k8s/openclaw/openclaw-instance.secret.yaml and b/k8s/openclaw/openclaw-instance.secret.yaml differ