diff --git a/docs/adr/0001-hybrid-mcp-tool-search-and-subagents.md b/docs/adr/0001-hybrid-mcp-tool-search-and-subagents.md index 2120c9b..7c96fdc 100644 --- a/docs/adr/0001-hybrid-mcp-tool-search-and-subagents.md +++ b/docs/adr/0001-hybrid-mcp-tool-search-and-subagents.md @@ -139,8 +139,7 @@ Subagent specifications, system prompts, tool assignments, and domain guidelines ### 3. Multi-Turn Continuity: Domain Stickiness & Rich State Bridge To prevent context loss during ongoing step-by-step dialogues (e.g. interactive cooking or workout design): -* **Domain Stickiness**: Follow-up questions on the active topic remain within the specialized subagent domain until an explicit topic shift occurs. -* **Rich State Bridge (``)**: Every subagent embeds a compact JSON state comment at the end of its response. The Conductor uses this state to answer trivial micro-checks in < 1s, or feeds it back into the subagent for complex modifications. +* **Persistent Context Continuity**: Follow-up turns are routed within the dedicated subagent topic session using native conversational memory, keeping user-facing messages clean without embedding raw JSON/HTML metadata tags. ### 4. 3-Tier Session Lifecycle (Preventing Unbounded Session Growth) 1. **Conversational Close**: Topic shifts or explicit user completion (*"Danke, das war's"*) reset the active domain pointer. diff --git a/k8s/openclaw/AGENTS.md b/k8s/openclaw/AGENTS.md index a2c9ec4..def862e 100644 --- a/k8s/openclaw/AGENTS.md +++ b/k8s/openclaw/AGENTS.md @@ -40,7 +40,7 @@ The multi-container pod structure is managed natively by the operator controller - 🏑🎬 `agents/grabengasse_media.md` (Topic 164 "Grabengasse & Media" β€” Home Assistant, Hue, Sonos, Brother Drucker, Jellyfin & Spotify) * **Inter-Agent Peer Escalation (A2A)**: Every subagent topic is configured with peer escalation authority via `sessions_spawn(agentId="", task="...")` to consult sibling agents when cross-domain expertise is needed. * **Modifying & Adding Agents**: To update prompt logic, tone, or tool instructions for any agent, edit its markdown file directly in `knowledge-wiki/agents/.md` and commit to git. No Kubernetes pod restarts required! -* **Multi-Turn Domain Stickiness**: When a conversation spans multiple turns in a subagent domain (e.g. refining a recipe or planning a trip), the Conductor detects domain continuity and routes to the same agent with the state bridge ``. +* **Multi-Turn Domain Stickiness**: When a conversation spans multiple turns in a subagent domain (e.g. refining a recipe or planning a trip), the Conductor detects domain continuity and routes to the same agent with clean conversational context. --- diff --git a/k8s/openclaw/README.md b/k8s/openclaw/README.md index f04e571..f7b00c3 100644 --- a/k8s/openclaw/README.md +++ b/k8s/openclaw/README.md @@ -23,7 +23,7 @@ The deployment is managed by the official **OpenClaw Operator** (`openclaw-opera * **Multi-Agent Specialist Delegation (ADR 0001)**: * **Conductor + 9 Subagents**: Lean front-of-house coordinator orchestrates 9 specialized domain subagents defined as code in the knowledge wiki (`agents/*.md`). * **Direct Fast Path (< 2s)**: Simple status, home automation toggles, and single calendar entries run directly in-place. - * **Domain Stickiness**: Multi-turn dialogue (e.g. step-by-step cooking or vacation planning) stays pinned to the active subagent with state carried over via ``. + * **Domain Stickiness**: Multi-turn dialogue (e.g. step-by-step cooking or vacation planning) stays pinned to the active subagent with persistent session continuity. * **Active MCP Servers**: `stadtrat` (ALLRIS Net), `supermarket` (Marktguru), `geizhals` (price histories), `ebay`, `garmin` (148 health tools), `bring` (4 lists), `spotify`, `calendar` (family + city), `jelly` (Jellyfin/Jellyseerr), `homeassistant`, `prometheus`, `gitea`, `weather`. --- diff --git a/k8s/openclaw/TROUBLESHOOTING.md b/k8s/openclaw/TROUBLESHOOTING.md index 5a9a943..d7a9691 100644 --- a/k8s/openclaw/TROUBLESHOOTING.md +++ b/k8s/openclaw/TROUBLESHOOTING.md @@ -25,6 +25,7 @@ This document captures fixed issues, operational quirks, and incident root-cause | **2026-10-05** | **CronJob & Alerting** | `openclaw-wiki-audit` CronJob failing daily with exit code 1; Prometheus alert `WikiAuditJobFailed` firing | `RESOLVED` | Appended `\|\| true` to `okf_lint.py` in `cronjob-wiki-audit.secret.yaml`, preventing `set -e` abort when lint findings exist. | | **2026-10-05** | **Cron Trigger State** | `rennerhof-website-watch` (Regensburg Topic 116) stuck in `status: error` with `ReferenceError: state is not defined` | `RESOLVED` | Cleared stale error state from `state_json` in `openclaw.sqlite` and triggered successful verification run. | | **2026-10-05** | **State Deadlock & Session Corruption** | Telegram Topic 117 (`#Recherche`) wedged with `restart recovery claim changed` and `entry_valid = 0` after crash | `RESOLVED` | Repaired `session_nodes` and `session_windows` (restored `entry_valid = 1`, synced `updated_at`, cleared dead claim), purged 20 obsolete cron sessions (>498k tokens), and added `--max-lines 200` compaction fallback. | +| **2026-10-06** | **Message Formatting & UX** | Subagents emitted raw HTML/XML comments (``) at the end of Telegram messages | `RESOLVED` | Removed obsolete `DOMAIN_STATE` State Bridge schema from all subagent prompts and Conductor config; replaced with clean Markdown output guardrails. | --- @@ -435,3 +436,39 @@ This document captures fixed issues, operational quirks, and incident root-cause - **How to Avoid Recurrence**: - The nightly maintenance cron now automatically purges stale cron sessions and pre-warms cache. - If a session ever reports `invalid persisted session row requires repair`, check `entry_valid == 1` and ensure `row.updated_at == record.updatedAt`. + +--- + +### Issue 15: Subagents Emitted Raw HTML/XML Comments (``) in Telegram Messages + +- **Date**: 2026-10-06 +- **Severity**: Medium (Visual/UX defect: raw HTML comments and JSON leaked into user-facing chat messages) +- **Symptom**: + Messages sent by specialist subagents in Telegram forum topics (`#Mila`, `#Medizin`, `#Shopping`, `#Travel`, `#Sport`, `#Regensburg`, etc.) consistently concluded with a raw HTML/XML comment block, for example: + ```html + + ``` +- **Root Cause**: + 1. **Prompt Directive**: In `knowledge-wiki/agents/*.md`, each subagent prompt contained a `## πŸ”„ State Bridge Schema` instructing the LLM: *β€žAt the end of your response, output a compact state comment: ``β€œ*. + 2. **Telegram Formatting Mismatch**: The schema was originally conceived in ADR 0001 under the assumption that `` would be invisible in chat rendering. However, Telegram's Bot API does not support hidden HTML comments. OpenClaw's text formatter safely escaped them to `<!-- ... -->`, causing the Telegram client to render the comments literally as visible text. + 3. **Architectural Redundancy**: Subagents operate inside persistent SQLite sessions with full conversation history and vector embeddings. Furthermore, structured domain states (sleep, nutrition, feeding, workouts) were transitioned to DuckDB/JSONL via `skills/datastore` (ADR 0004). No system component or code read `DOMAIN_STATE`. +- **Resolution**: + 1. Removed `## πŸ”„ State Bridge Schema` from all 9 subagent definition files in `knowledge-wiki/agents/*.md` (`hebamme`, `doktor`, `shopping`, `travel`, `fitness`, `regensburg`, `recherche`, `itguy`, `grabengasse_media`, and `README.md`). + 2. Replaced the directive with an explicit clean-output guardrail: + ```markdown + ## πŸ’¬ Output Formatting & Telegram Guardrails + - Output clean Markdown only for Telegram. + - NEVER append machine-readable metadata, HTML comments (``), XML tags, or raw JSON payloads to your chat responses. + ``` + 3. Removed the `Pass State Bridge ()` clause from the Conductor system prompt in `k8s/openclaw/openclaw-instance.secret.yaml`. + 4. Pushed updates to `knowledge-wiki` git repository, pulled latest into `/home/node/.openclaw/wiki/main`, and synchronized live workspace files in `/home/node/.openclaw/workspace-*/AGENTS.md`. + 5. Applied updated `openclaw-instance.secret.yaml` to Kubernetes. +- **How to Avoid Recurrence**: + - Never instruct LLMs to embed machine-readable metadata or pseudo-comments (``) in user-facing messenger outputs. + - Rely on persistent session context and structured stores (`skills/datastore`) for state retention. + diff --git a/k8s/openclaw/openclaw-instance.secret.yaml b/k8s/openclaw/openclaw-instance.secret.yaml index e2882cf..71e06ab 100644 Binary files a/k8s/openclaw/openclaw-instance.secret.yaml and b/k8s/openclaw/openclaw-instance.secret.yaml differ diff --git a/k8s/openclaw/setup-compaction-cron.sh b/k8s/openclaw/setup-compaction-cron.sh index ccf115a..a5104da 100755 --- a/k8s/openclaw/setup-compaction-cron.sh +++ b/k8s/openclaw/setup-compaction-cron.sh @@ -63,6 +63,7 @@ def prune_stale_cron_sessions(): agent = db.split("/")[5] try: conn = sqlite3.connect(db, timeout=30.0) + conn.execute("PRAGMA foreign_keys = ON;") cursor = conn.cursor() cursor.execute("SELECT name FROM sqlite_master WHERE type=\"table\"") all_tables = set(r[0] for r in cursor.fetchall()) @@ -82,7 +83,7 @@ def prune_stale_cron_sessions(): if stale: tokens_reclaimed = sum(t[2] for t in stale) for k, sid, _ in stale: - for t in ["transcript_events", "transcript_event_identities", "session_transcript_active_events", "transcript_rewrite_watermarks", "session_transcript_index_state", "session_transcript_fts", "session_conversations"]: + for t in ["trajectory_runtime_events", "transcript_events", "transcript_event_identities", "session_transcript_active_events", "transcript_rewrite_watermarks", "session_transcript_index_state", "session_transcript_fts", "session_conversations"]: if t in all_tables and sid: cursor.execute(f"DELETE FROM {t} WHERE session_id = ?", (sid,)) for t in ["session_participants", "heartbeat_outcomes", "session_progress_cards", "board_widgets", "board_tabs"]: @@ -90,25 +91,39 @@ def prune_stale_cron_sessions(): cursor.execute(f"DELETE FROM {t} WHERE session_key = ?", (k,)) if "memory_entry_origins" in all_tables: cursor.execute("DELETE FROM memory_entry_origins WHERE session_key = ? OR session_id = ?", (k, sid or "")) + if "session_windows" in all_tables: + cursor.execute("DELETE FROM session_windows WHERE session_key = ? OR session_id = ?", (k, sid or "")) cursor.execute("DELETE FROM session_nodes WHERE session_key = ?", (k,)) - - # Resolve any dangling foreign key references - while True: - cursor.execute("PRAGMA foreign_key_check;") - violations = cursor.fetchall() - if not violations: - break - for table, rowid, parent, fkid in violations: - cursor.execute(f"DELETE FROM {table} WHERE rowid = ?", (rowid,)) - conn.commit() total_pruned += len(stale) total_tokens_reclaimed += tokens_reclaimed print(f" [{agent}] Pruned {len(stale)} dead cron sessions ({tokens_reclaimed} tokens reclaimed)") + + # Unconditionally resolve any dangling foreign key references across all tables + while True: + cursor.execute("PRAGMA foreign_key_check;") + violations = cursor.fetchall() + if not violations: + break + for table, rowid, parent, fkid in violations: + cursor.execute(f"DELETE FROM {table} WHERE rowid = ?", (rowid,)) + conn.commit() conn.close() except Exception as e: print(f" [{agent}] Pruning error: {e}") + # Clear resolved quarantine entries if all databases are healthy + try: + q_path = "/home/node/.openclaw/state/openclaw-quarantine.sqlite" + if os.path.exists(q_path): + q_conn = sqlite3.connect(q_path) + q_cur = q_conn.cursor() + q_cur.execute("DELETE FROM quarantined_databases") + q_conn.commit() + q_conn.close() + except Exception as e: + print(f"Quarantine cleanup error: {e}") + print(f"Cron pruning complete: {total_pruned} sessions pruned, {total_tokens_reclaimed} tokens reclaimed.") def compact_active_sessions(): diff --git a/knowledge-wiki b/knowledge-wiki index 1397837..55c809d 160000 --- a/knowledge-wiki +++ b/knowledge-wiki @@ -1 +1 @@ -Subproject commit 139783762361e3c755009606d998f103690bea9e +Subproject commit 55c809dd792b5809a2678c3c306070275a2e2ff9