feat(openclaw): strip tracking agent prompt bloat and add 15m deterministic sliding-window compaction
- Disable bootstrap context injection (contextInjection: never) and skill catalog injection (skills: []) on mila_tracker and nutrition_tracker, dropping baseline prompt by 95% (<350 tokens) - Disable reasoning loops on tracking agents (thinkingDefault: off) for instantaneous sub-second execution - Add compact_tracking_sessions.py with deterministic 8-line sliding window (--max-lines 8) requiring 0 LLM tokens - Register automated 15-minute cron job (tracking-session-sliding-window) in OpenClaw gateway - Streamline setup-compaction-cron.sh to sync python scripts directly and manage both 15m and nightly maintenance
This commit is contained in:
@@ -94,12 +94,30 @@ This repository is divided into distinct domains, each with its own `AGENTS.md`
|
|||||||
* **Key Tools**: Interactive `uv run flash.py` wizard and `./send_message.sh` CLI push messaging.
|
* **Key Tools**: Interactive `uv run flash.py` wizard and `./send_message.sh` CLI push messaging.
|
||||||
* **Read**: [projects/esphome_display_kitchen/AGENTS.md](./projects/esphome_display_kitchen/AGENTS.md)
|
* **Read**: [projects/esphome_display_kitchen/AGENTS.md](./projects/esphome_display_kitchen/AGENTS.md)
|
||||||
|
|
||||||
### 12. Utility Scripts (`scripts/`)
|
### 12. [DuckDB Backend](./projects/duckdb-backend/AGENTS.md) (`projects/duckdb-backend/`)
|
||||||
|
* **Purpose**: High-performance HTTP analytics and event tracking data service executing in-memory DuckDB queries against Knowledge Wiki JSONL datasets.
|
||||||
|
* **Manifests**: `k8s/family/` (`duckdb-backend.yaml`, `pvc.yaml`, `ingress.yaml` in namespace `family`).
|
||||||
|
* **Endpoints**: Curated dashboards (`/duckdb-api/v1/mila/dashboard`, `/duckdb-api/v1/moritz/dashboard`), generic SQL queries (`/duckdb-api/v1/query`), and event tracking writes with Git sync.
|
||||||
|
* **Read**: [projects/duckdb-backend/AGENTS.md](./projects/duckdb-backend/AGENTS.md)
|
||||||
|
|
||||||
|
### 13. [Mila Tracking Web App](./projects/mila/AGENTS.md) (`projects/mila/`)
|
||||||
|
* **Purpose**: Reactive mobile-first PWA for Mila's baby sleep, feeding, diaper tracking, and growth percentiles (`https://family.haumdaucher.de/tracking/mila`).
|
||||||
|
* **Manifest**: `k8s/family/mila.yaml` (Namespace: `family`).
|
||||||
|
* **Key Rules**: Vue 3, Vite, TypeScript, and Tailwind CSS; live wake timer; interactive 1-tap and custom-time tracking modal.
|
||||||
|
* **Read**: [projects/mila/AGENTS.md](./projects/mila/AGENTS.md)
|
||||||
|
|
||||||
|
### 14. [Moritz Health & Nutrition Web App](./projects/moritz/AGENTS.md) (`projects/moritz/`)
|
||||||
|
* **Purpose**: Apple Health-inspired nutrition and metabolic balance dashboard (`https://family.haumdaucher.de/tracking/moritz`).
|
||||||
|
* **Manifest**: `k8s/family/moritz.yaml` (Namespace: `family`).
|
||||||
|
* **Key Rules**: Vue 3, Vite, TypeScript, and Tailwind CSS; net calorie balance hero, Garmin health breakdown, and quick meal logger.
|
||||||
|
* **Read**: [projects/moritz/AGENTS.md](./projects/moritz/AGENTS.md)
|
||||||
|
|
||||||
|
### 15. Utility Scripts (`scripts/`)
|
||||||
* **Purpose**: Cross-cutting operational and administration CLI scripts that do not belong inside Kubernetes namespace directories.
|
* **Purpose**: Cross-cutting operational and administration CLI scripts that do not belong inside Kubernetes namespace directories.
|
||||||
* **Key Tools**:
|
* **Key Tools**:
|
||||||
* `scripts/htpasswd.py`: Generates bcrypt-hashed htpasswd credentials for ingress basic auth secrets.
|
* `scripts/htpasswd.py`: Generates bcrypt-hashed htpasswd credentials for ingress basic auth secrets.
|
||||||
|
|
||||||
### 13. Network & Router Integration (`fritzbox/`)
|
### 16. Network & Router Integration (`fritzbox/`)
|
||||||
* **Purpose**: Site-to-Site WireGuard LAN coupling and DynDNS OVH synchronization for the home FritzBox router.
|
* **Purpose**: Site-to-Site WireGuard LAN coupling and DynDNS OVH synchronization for the home FritzBox router.
|
||||||
* **Cluster Endpoint**: `k8s/wireguard/` (`vpn.haumdaucher.de`).
|
* **Cluster Endpoint**: `k8s/wireguard/` (`vpn.haumdaucher.de`).
|
||||||
* **Read**: [fritzbox/README.md](./fritzbox/README.md)
|
* **Read**: [fritzbox/README.md](./fritzbox/README.md)
|
||||||
|
|||||||
Binary file not shown.
@@ -23,6 +23,15 @@ THRESHOLD_EVENT_COUNT = 30
|
|||||||
CRON_MAX_AGE_HOURS = 48
|
CRON_MAX_AGE_HOURS = 48
|
||||||
RPC_TIMEOUT_MS = 180000
|
RPC_TIMEOUT_MS = 180000
|
||||||
|
|
||||||
|
# Tracking Channels: Deterministic sliding-window (mila_tracker, nutrition_tracker).
|
||||||
|
# Strict sliding window of ~5-8 events (2-3 turns max), 0 LLM tokens, instant truncation.
|
||||||
|
TRACKING_TARGETS = [
|
||||||
|
("mila_tracker", "agent:mila_tracker:telegram:group:-1004326921510:topic:2796"),
|
||||||
|
("nutrition_tracker", "agent:nutrition_tracker:telegram:group:-1004326921510:topic:2797"),
|
||||||
|
]
|
||||||
|
MAX_TRACKING_EVENTS = 8
|
||||||
|
MAX_TRACKING_LINES = 8
|
||||||
|
|
||||||
# Tier A: High-frequency interaction topics (koch, hebamme, shopping).
|
# Tier A: High-frequency interaction topics (koch, hebamme, shopping).
|
||||||
# Deep-compaction keeps a compact summary + latest 2 turns without completely wiping conversational memory.
|
# Deep-compaction keeps a compact summary + latest 2 turns without completely wiping conversational memory.
|
||||||
TIER_A_TARGETS = [
|
TIER_A_TARGETS = [
|
||||||
@@ -117,6 +126,41 @@ def prune_stale_cron_sessions():
|
|||||||
|
|
||||||
print(f"Cron pruning complete: {total_pruned} sessions pruned, {total_tokens_reclaimed} tokens reclaimed.")
|
print(f"Cron pruning complete: {total_pruned} sessions pruned, {total_tokens_reclaimed} tokens reclaimed.")
|
||||||
|
|
||||||
|
def compact_tracking_sessions():
|
||||||
|
print("=== Tracking Channels Sliding-Window Maintenance (mila_tracker, nutrition_tracker) ===")
|
||||||
|
for agent, key in TRACKING_TARGETS:
|
||||||
|
db_path = f"/home/node/.openclaw/agents/{agent}/agent/openclaw-agent.sqlite"
|
||||||
|
if not os.path.exists(db_path):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
con = sqlite3.connect(db_path, timeout=5.0)
|
||||||
|
row = con.execute("SELECT current_session_id FROM session_nodes WHERE session_key=?", (key,)).fetchone()
|
||||||
|
if not row or not row[0]:
|
||||||
|
con.close()
|
||||||
|
print(f" [{agent}] No active session node found for {key}.")
|
||||||
|
continue
|
||||||
|
sid = row[0]
|
||||||
|
cnt = con.execute("SELECT count(*) FROM transcript_events WHERE session_id=?", (sid,)).fetchone()[0]
|
||||||
|
con.close()
|
||||||
|
|
||||||
|
if cnt > MAX_TRACKING_EVENTS:
|
||||||
|
print(f" [{agent}] Session {key} has {cnt} events (> {MAX_TRACKING_EVENTS}). Truncating to last {MAX_TRACKING_LINES} lines...")
|
||||||
|
cmd = [
|
||||||
|
"openclaw", "sessions", "compact", key,
|
||||||
|
"--agent", agent,
|
||||||
|
"--max-lines", str(MAX_TRACKING_LINES),
|
||||||
|
"--timeout", "30000"
|
||||||
|
]
|
||||||
|
r = subprocess.run(cmd, capture_output=True, text=True)
|
||||||
|
if r.returncode == 0:
|
||||||
|
print(f" ✓ Truncated successfully: {r.stdout.strip()}")
|
||||||
|
else:
|
||||||
|
print(f" ✗ Truncation failed: {r.stderr.strip()}")
|
||||||
|
else:
|
||||||
|
print(f" [{agent}] Session {key} has {cnt} events (<= {MAX_TRACKING_EVENTS}). Clean.")
|
||||||
|
except Exception as e:
|
||||||
|
print(f" ✗ [{agent}] Error during tracking compaction: {e}")
|
||||||
|
|
||||||
def compact_tier_a_sessions():
|
def compact_tier_a_sessions():
|
||||||
print("=== Tier A: Deep-Compaction (koch, hebamme, shopping) ===")
|
print("=== Tier A: Deep-Compaction (koch, hebamme, shopping) ===")
|
||||||
for agent, key in TIER_A_TARGETS:
|
for agent, key in TIER_A_TARGETS:
|
||||||
@@ -180,7 +224,7 @@ def get_session_metrics(agent, key, sid=None):
|
|||||||
return 0, 0
|
return 0, 0
|
||||||
|
|
||||||
def compact_active_sessions():
|
def compact_active_sessions():
|
||||||
tier_a_keys = set(t[1] for t in TIER_A_TARGETS)
|
excluded_keys = set(t[1] for t in TIER_A_TARGETS) | set(t[1] for t in TRACKING_TARGETS)
|
||||||
print(f"=== Tier B: Running Multi-Agent Compaction (threshold: {THRESHOLD_TOKENS} tokens / {THRESHOLD_TRANSCRIPT_BYTES//1024}KB, LLM summarization) ===")
|
print(f"=== Tier B: Running Multi-Agent Compaction (threshold: {THRESHOLD_TOKENS} tokens / {THRESHOLD_TRANSCRIPT_BYTES//1024}KB, LLM summarization) ===")
|
||||||
res = subprocess.run(["openclaw", "sessions", "--all-agents", "--json"], capture_output=True, text=True)
|
res = subprocess.run(["openclaw", "sessions", "--all-agents", "--json"], capture_output=True, text=True)
|
||||||
if res.returncode != 0:
|
if res.returncode != 0:
|
||||||
@@ -197,8 +241,8 @@ def compact_active_sessions():
|
|||||||
tokens = s.get("totalTokens") or 0
|
tokens = s.get("totalTokens") or 0
|
||||||
sid = s.get("sessionId")
|
sid = s.get("sessionId")
|
||||||
|
|
||||||
# Skip Tier A agents as they are handled via hard-reset
|
# Skip Tier A and Tracking agents as they have dedicated routines
|
||||||
if key in tier_a_keys:
|
if key in excluded_keys:
|
||||||
continue
|
continue
|
||||||
|
|
||||||
# Target main sessions and all telegram sessions (topics and direct)
|
# Target main sessions and all telegram sessions (topics and direct)
|
||||||
@@ -280,6 +324,7 @@ def main():
|
|||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
prune_stale_cron_sessions()
|
prune_stale_cron_sessions()
|
||||||
|
compact_tracking_sessions()
|
||||||
compact_tier_a_sessions()
|
compact_tier_a_sessions()
|
||||||
compact_active_sessions()
|
compact_active_sessions()
|
||||||
checkpoint_wal_journals()
|
checkpoint_wal_journals()
|
||||||
|
|||||||
@@ -0,0 +1,66 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Lightweight sliding-window compaction for OpenClaw tracking channels:
|
||||||
|
- mila_tracker (topic 2796)
|
||||||
|
- nutrition_tracker (topic 2797)
|
||||||
|
|
||||||
|
Maintains a strict sliding window of ~5-8 transcript events (max 2-3 turns).
|
||||||
|
Uses native gateway truncation (--max-lines 8) requiring 0 LLM tokens and 0 external API calls.
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
import sqlite3
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
|
||||||
|
MAX_TRACKING_EVENTS = 8
|
||||||
|
MAX_TRACKING_LINES = 8
|
||||||
|
TIMEOUT_MS = 30000
|
||||||
|
|
||||||
|
TRACKING_TARGETS = [
|
||||||
|
("mila_tracker", "agent:mila_tracker:telegram:group:-1004326921510:topic:2796"),
|
||||||
|
("nutrition_tracker", "agent:nutrition_tracker:telegram:group:-1004326921510:topic:2797"),
|
||||||
|
]
|
||||||
|
|
||||||
|
def check_and_compact_tracking():
|
||||||
|
compacted_count = 0
|
||||||
|
skipped_count = 0
|
||||||
|
|
||||||
|
for agent, key in TRACKING_TARGETS:
|
||||||
|
db_path = f"/home/node/.openclaw/agents/{agent}/agent/openclaw-agent.sqlite"
|
||||||
|
if not os.path.exists(db_path):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
con = sqlite3.connect(db_path, timeout=5.0)
|
||||||
|
row = con.execute("SELECT current_session_id FROM session_nodes WHERE session_key=?", (key,)).fetchone()
|
||||||
|
if not row or not row[0]:
|
||||||
|
con.close()
|
||||||
|
continue
|
||||||
|
sid = row[0]
|
||||||
|
cnt = con.execute("SELECT count(*) FROM transcript_events WHERE session_id=?", (sid,)).fetchone()[0]
|
||||||
|
con.close()
|
||||||
|
|
||||||
|
if cnt > MAX_TRACKING_EVENTS:
|
||||||
|
print(f"[{agent}] Session {key} has {cnt} events (threshold {MAX_TRACKING_EVENTS}). Truncating to last {MAX_TRACKING_LINES} lines...")
|
||||||
|
cmd = [
|
||||||
|
"openclaw", "sessions", "compact", key,
|
||||||
|
"--agent", agent,
|
||||||
|
"--max-lines", str(MAX_TRACKING_LINES),
|
||||||
|
"--timeout", str(TIMEOUT_MS)
|
||||||
|
]
|
||||||
|
res = subprocess.run(cmd, capture_output=True, text=True)
|
||||||
|
if res.returncode == 0:
|
||||||
|
print(f" ✓ [{agent}] Truncated successfully: {res.stdout.strip()}")
|
||||||
|
compacted_count += 1
|
||||||
|
else:
|
||||||
|
print(f" ✗ [{agent}] Truncation failed: {res.stderr.strip()}", file=sys.stderr)
|
||||||
|
else:
|
||||||
|
skipped_count += 1
|
||||||
|
except Exception as e:
|
||||||
|
print(f"[{agent}] Error checking session: {e}", file=sys.stderr)
|
||||||
|
|
||||||
|
if compacted_count > 0:
|
||||||
|
print(f"Tracking maintenance: {compacted_count} sessions truncated, {skipped_count} within window.")
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
check_and_compact_tracking()
|
||||||
@@ -1,14 +1,16 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# ==============================================================================
|
# ==============================================================================
|
||||||
# setup-compaction-cron.sh
|
# setup-compaction-cron.sh
|
||||||
# Declaratively registers the Multi-Agent Nightly Session Compaction in OpenClaw.
|
# Declaratively registers OpenClaw session maintenance automations:
|
||||||
# Schedule: Every night at 04:45 Europe/Berlin (02:45 UTC) (cron: 45 4 * * *)
|
# 1. Tracking Sliding Window (every 15m): Truncates mila_tracker & nutrition_tracker
|
||||||
#
|
# to <= 8 events (~5 messages / 2 turns) via instant native gateway truncation.
|
||||||
# Architecture:
|
# 2. Nightly Deep Compaction (every night at 04:45 Europe/Berlin):
|
||||||
# - Tier A: Deep-Compaction for high-frequency topics (koch, hebamme, shopping)
|
# - Pre-warms SQLite page cache on rotational storage
|
||||||
# (maintains conversational continuity while keeping token count very low)
|
# - Recovers wedged/quarantined sessions
|
||||||
# - Tier B: Smart Compaction for long-running project agents (itguy, recherche, regensburg, fitness, doktor)
|
# - Prunes abandoned isolated cron sessions (>48h)
|
||||||
# - WAL checkpoints and SQLite optimization without blocking full VACUUM
|
# - Deep-compacts conversational topics (koch, hebamme, shopping)
|
||||||
|
# - Smart-compacts active long-running sessions (itguy, recherche, etc.)
|
||||||
|
# - Flushes SQLite WAL journals and executes PRAGMA optimize
|
||||||
# ==============================================================================
|
# ==============================================================================
|
||||||
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
@@ -16,326 +18,62 @@ set -euo pipefail
|
|||||||
NAMESPACE="${NAMESPACE:-openclaw}"
|
NAMESPACE="${NAMESPACE:-openclaw}"
|
||||||
POD_NAME="${POD_NAME:-openclaw-0}"
|
POD_NAME="${POD_NAME:-openclaw-0}"
|
||||||
CONTAINER_NAME="${CONTAINER_NAME:-openclaw}"
|
CONTAINER_NAME="${CONTAINER_NAME:-openclaw}"
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
|
||||||
JOB_NAME="nightly-session-compaction"
|
echo "=== Registering Multi-Agent Session Maintenance in OpenClaw ($POD_NAME) ==="
|
||||||
|
|
||||||
|
# 1. Install maintenance scripts from repo onto PVC
|
||||||
|
echo "Syncing compact_tracking_sessions.py to pod..."
|
||||||
|
kubectl exec -i -n "$NAMESPACE" "$POD_NAME" -c "$CONTAINER_NAME" -- /bin/sh -c \
|
||||||
|
'cat > /home/node/.openclaw/scripts/compact_tracking_sessions.py && chmod +x /home/node/.openclaw/scripts/compact_tracking_sessions.py' \
|
||||||
|
< "$SCRIPT_DIR/scripts/compact_tracking_sessions.py"
|
||||||
|
|
||||||
|
echo "Syncing compact_sessions.py to pod..."
|
||||||
|
kubectl exec -i -n "$NAMESPACE" "$POD_NAME" -c "$CONTAINER_NAME" -- /bin/sh -c \
|
||||||
|
'cat > /home/node/.openclaw/scripts/compact_sessions.py && chmod +x /home/node/.openclaw/scripts/compact_sessions.py' \
|
||||||
|
< "$SCRIPT_DIR/scripts/compact_sessions.py"
|
||||||
|
|
||||||
|
# 2. Register / Update 15-Minute Tracking Sliding Window Automation
|
||||||
|
TRACKING_JOB="tracking-session-sliding-window"
|
||||||
|
echo "Checking existing OpenClaw Cron automation '$TRACKING_JOB'..."
|
||||||
|
EXISTING_TRACKING_ID=$(kubectl exec -n "$NAMESPACE" "$POD_NAME" -c "$CONTAINER_NAME" -- openclaw cron list --json 2>/dev/null | python3 -c 'import sys, json; data=json.load(sys.stdin); print(next((j["id"] for j in data.get("jobs", []) if j.get("declarationKey") == "'"$TRACKING_JOB"'"), ""))' 2>/dev/null || true)
|
||||||
|
|
||||||
|
if [ -n "$EXISTING_TRACKING_ID" ]; then
|
||||||
|
echo "Updating existing tracking sliding window automation '$EXISTING_TRACKING_ID' (every 15m)..."
|
||||||
|
kubectl exec -n "$NAMESPACE" "$POD_NAME" -c "$CONTAINER_NAME" -- openclaw cron edit "$EXISTING_TRACKING_ID" \
|
||||||
|
--every "15m" \
|
||||||
|
--command "python3 /home/node/.openclaw/scripts/compact_tracking_sessions.py"
|
||||||
|
else
|
||||||
|
echo "Registering new tracking sliding window automation '$TRACKING_JOB' (every 15m)..."
|
||||||
|
kubectl exec -n "$NAMESPACE" "$POD_NAME" -c "$CONTAINER_NAME" -- openclaw cron add \
|
||||||
|
--name "Tracking Session Sliding Window" \
|
||||||
|
--declaration-key "$TRACKING_JOB" \
|
||||||
|
--description "Lightweight sliding-window truncation (max 8 events) for mila_tracker and nutrition_tracker" \
|
||||||
|
--every "15m" \
|
||||||
|
--session "isolated" \
|
||||||
|
--no-deliver \
|
||||||
|
--command "python3 /home/node/.openclaw/scripts/compact_tracking_sessions.py"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 3. Register / Update Nightly Deep Compaction Automation
|
||||||
|
NIGHTLY_JOB="nightly-session-compaction"
|
||||||
CRON_SCHEDULE="45 4 * * *"
|
CRON_SCHEDULE="45 4 * * *"
|
||||||
TIMEZONE="Europe/Berlin"
|
TIMEZONE="Europe/Berlin"
|
||||||
|
|
||||||
echo "=== Registering Multi-Agent Nightly Session Maintenance in OpenClaw ($POD_NAME) ==="
|
echo "Checking existing OpenClaw Cron automation '$NIGHTLY_JOB'..."
|
||||||
|
EXISTING_NIGHTLY_ID=$(kubectl exec -n "$NAMESPACE" "$POD_NAME" -c "$CONTAINER_NAME" -- openclaw cron list --json 2>/dev/null | python3 -c 'import sys, json; data=json.load(sys.stdin); print(next((j["id"] for j in data.get("jobs", []) if j.get("declarationKey") == "'"$NIGHTLY_JOB"'"), ""))' 2>/dev/null || true)
|
||||||
|
|
||||||
# 1. Install or ensure python maintenance script is present on PVC
|
if [ -n "$EXISTING_NIGHTLY_ID" ]; then
|
||||||
echo "Ensuring /home/node/.openclaw/scripts/compact_sessions.py exists..."
|
echo "Updating existing nightly compaction automation '$EXISTING_NIGHTLY_ID' ($CRON_SCHEDULE @ $TIMEZONE)..."
|
||||||
kubectl exec -i -n "$NAMESPACE" "$POD_NAME" -c "$CONTAINER_NAME" -- /bin/sh -c 'cat << "EOF" > /home/node/.openclaw/scripts/compact_sessions.py
|
kubectl exec -n "$NAMESPACE" "$POD_NAME" -c "$CONTAINER_NAME" -- openclaw cron edit "$EXISTING_NIGHTLY_ID" \
|
||||||
#!/usr/bin/env python3
|
|
||||||
"""
|
|
||||||
Nightly multi-agent session maintenance and compaction script for OpenClaw.
|
|
||||||
1. Pre-warms OS page cache for SQLite databases on spinning disks.
|
|
||||||
2. Recovers any wedged or locked sessions via recover_sessions.py.
|
|
||||||
3. Prunes abandoned isolated cron sessions (>48h old) to prevent token/state bloat.
|
|
||||||
4. Tier A Hard-Reset: Resets koch, hebamme, and shopping topics to 0 tokens (clean archive).
|
|
||||||
5. Tier B Smart-Compaction: Compacts active sessions exceeding token thresholds (5s pacing).
|
|
||||||
6. Checkpoints SQLite WAL journals across all agent databases to disk.
|
|
||||||
7. Optimizes SQLite query planner without blocking full VACUUM.
|
|
||||||
"""
|
|
||||||
import glob
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import sqlite3
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import time
|
|
||||||
|
|
||||||
THRESHOLD_TOKENS = 15000
|
|
||||||
THRESHOLD_TRANSCRIPT_BYTES = 200000
|
|
||||||
THRESHOLD_EVENT_COUNT = 30
|
|
||||||
CRON_MAX_AGE_HOURS = 48
|
|
||||||
RPC_TIMEOUT_MS = 180000
|
|
||||||
|
|
||||||
# Tier A: High-frequency interaction and tracking topics (koch, hebamme, shopping, mila_tracker, nutrition_tracker).
|
|
||||||
# Deep-compaction keeps a compact summary + latest 2 turns without completely wiping conversational memory.
|
|
||||||
TIER_A_TARGETS = [
|
|
||||||
("koch", "agent:koch:telegram:group:-1004326921510:topic:21"),
|
|
||||||
("hebamme", "agent:hebamme:telegram:group:-1004326921510:topic:22"),
|
|
||||||
("shopping", "agent:shopping:telegram:group:-1004326921510:topic:115"),
|
|
||||||
("mila_tracker", "agent:mila_tracker:telegram:group:-1004326921510:topic:2796"),
|
|
||||||
("nutrition_tracker", "agent:nutrition_tracker:telegram:group:-1004326921510:topic:2797"),
|
|
||||||
]
|
|
||||||
|
|
||||||
def prewarm_page_cache():
|
|
||||||
print("=== Pre-warming SQLite databases into OS page cache ===")
|
|
||||||
t0 = time.time()
|
|
||||||
try:
|
|
||||||
subprocess.run(
|
|
||||||
"cat /home/node/.openclaw/state/openclaw.sqlite /home/node/.openclaw/agents/*/agent/openclaw-agent.sqlite > /dev/null 2>&1 || true",
|
|
||||||
shell=True,
|
|
||||||
timeout=90.0
|
|
||||||
)
|
|
||||||
print(f"Page cache pre-warmed in {time.time() - t0:.2f}s")
|
|
||||||
except Exception as e:
|
|
||||||
print(f"Page cache pre-warming warning: {e}")
|
|
||||||
|
|
||||||
def prune_stale_cron_sessions():
|
|
||||||
print(f"=== Pruning isolated cron sessions older than {CRON_MAX_AGE_HOURS}h ===")
|
|
||||||
cutoff_ms = int(time.time() * 1000) - (CRON_MAX_AGE_HOURS * 3600 * 1000)
|
|
||||||
total_pruned = 0
|
|
||||||
total_tokens_reclaimed = 0
|
|
||||||
|
|
||||||
for db in sorted(glob.glob("/home/node/.openclaw/agents/*/agent/openclaw-agent.sqlite")):
|
|
||||||
agent = db.split("/")[5]
|
|
||||||
try:
|
|
||||||
conn = sqlite3.connect(db, timeout=30.0)
|
|
||||||
conn.execute("PRAGMA foreign_keys = ON;")
|
|
||||||
cursor = conn.cursor()
|
|
||||||
cursor.execute("SELECT name FROM sqlite_master WHERE type=\"table\"")
|
|
||||||
all_tables = set(r[0] for r in cursor.fetchall())
|
|
||||||
|
|
||||||
cursor.execute("SELECT session_key, current_session_id, updated_at, entry_json FROM session_nodes WHERE session_key LIKE \"%:cron:%\"")
|
|
||||||
stale = []
|
|
||||||
for k, sid, up, ej in cursor.fetchall():
|
|
||||||
if up and up < cutoff_ms:
|
|
||||||
tok = 0
|
|
||||||
if ej:
|
|
||||||
try:
|
|
||||||
tok = json.loads(ej).get("totalTokens", 0) or 0
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
stale.append((k, sid, tok))
|
|
||||||
|
|
||||||
if stale:
|
|
||||||
tokens_reclaimed = sum(t[2] for t in stale)
|
|
||||||
for k, sid, _ in stale:
|
|
||||||
for t in ["trajectory_runtime_events", "transcript_events", "transcript_event_identities", "session_transcript_active_events", "transcript_rewrite_watermarks", "session_transcript_index_state", "session_transcript_fts", "session_conversations"]:
|
|
||||||
if t in all_tables and sid:
|
|
||||||
cursor.execute(f"DELETE FROM {t} WHERE session_id = ?", (sid,))
|
|
||||||
for t in ["session_participants", "heartbeat_outcomes", "session_progress_cards", "board_widgets", "board_tabs"]:
|
|
||||||
if t in all_tables:
|
|
||||||
cursor.execute(f"DELETE FROM {t} WHERE session_key = ?", (k,))
|
|
||||||
if "memory_entry_origins" in all_tables:
|
|
||||||
cursor.execute("DELETE FROM memory_entry_origins WHERE session_key = ? OR session_id = ?", (k, sid or ""))
|
|
||||||
if "session_windows" in all_tables:
|
|
||||||
cursor.execute("DELETE FROM session_windows WHERE session_key = ? OR session_id = ?", (k, sid or ""))
|
|
||||||
cursor.execute("DELETE FROM session_nodes WHERE session_key = ?", (k,))
|
|
||||||
conn.commit()
|
|
||||||
total_pruned += len(stale)
|
|
||||||
total_tokens_reclaimed += tokens_reclaimed
|
|
||||||
print(f" [{agent}] Pruned {len(stale)} dead cron sessions ({tokens_reclaimed} tokens reclaimed)")
|
|
||||||
|
|
||||||
# Unconditionally resolve any dangling foreign key references across all tables
|
|
||||||
while True:
|
|
||||||
cursor.execute("PRAGMA foreign_key_check;")
|
|
||||||
violations = cursor.fetchall()
|
|
||||||
if not violations:
|
|
||||||
break
|
|
||||||
for table, rowid, parent, fkid in violations:
|
|
||||||
cursor.execute(f"DELETE FROM {table} WHERE rowid = ?", (rowid,))
|
|
||||||
conn.commit()
|
|
||||||
conn.close()
|
|
||||||
except Exception as e:
|
|
||||||
print(f" [{agent}] Pruning error: {e}")
|
|
||||||
|
|
||||||
# Clear resolved quarantine entries if all databases are healthy
|
|
||||||
try:
|
|
||||||
q_path = "/home/node/.openclaw/state/openclaw-quarantine.sqlite"
|
|
||||||
if os.path.exists(q_path):
|
|
||||||
q_conn = sqlite3.connect(q_path)
|
|
||||||
q_cur = q_conn.cursor()
|
|
||||||
q_cur.execute("DELETE FROM quarantined_databases")
|
|
||||||
q_conn.commit()
|
|
||||||
q_conn.close()
|
|
||||||
except Exception as e:
|
|
||||||
print(f"Quarantine cleanup error: {e}")
|
|
||||||
|
|
||||||
print(f"Cron pruning complete: {total_pruned} sessions pruned, {total_tokens_reclaimed} tokens reclaimed.")
|
|
||||||
|
|
||||||
def compact_tier_a_sessions():
|
|
||||||
print("=== Tier A: Deep-Compaction (koch, hebamme, shopping) ===")
|
|
||||||
for agent, key in TIER_A_TARGETS:
|
|
||||||
db_path = f"/home/node/.openclaw/agents/{agent}/agent/openclaw-agent.sqlite"
|
|
||||||
if not os.path.exists(db_path):
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
con = sqlite3.connect(db_path, timeout=10.0)
|
|
||||||
row = con.execute("SELECT current_session_id FROM session_nodes WHERE session_key=?", (key,)).fetchone()
|
|
||||||
con.close()
|
|
||||||
if not row or not row[0]:
|
|
||||||
print(f" [Tier A] {agent} ({key}) already at 0 (no active session).")
|
|
||||||
continue
|
|
||||||
ev_cnt, ev_bytes = get_session_metrics(agent, key, row[0])
|
|
||||||
if ev_cnt <= 2 and ev_bytes < 20000:
|
|
||||||
print(f" [Tier A] {agent} ({key}) has only {ev_cnt} events ({ev_bytes} bytes), skipping.")
|
|
||||||
continue
|
|
||||||
print(f" [Tier A] Deep-compacting {agent} ({key}) [{ev_cnt} events, {ev_bytes//1024}KB]...")
|
|
||||||
r = subprocess.run(
|
|
||||||
["openclaw", "sessions", "compact", key, "--agent", agent, "--timeout", str(RPC_TIMEOUT_MS)],
|
|
||||||
capture_output=True, text=True
|
|
||||||
)
|
|
||||||
if r.returncode == 0:
|
|
||||||
print(f" ✓ Deep-compacted: {r.stdout.strip()}")
|
|
||||||
elif "Already compacted" in r.stderr or "Already compacted" in r.stdout:
|
|
||||||
print(f" ✓ Already compacted.")
|
|
||||||
else:
|
|
||||||
print(f" ✗ Standard compact error: {r.stderr.strip()}; retrying with --max-lines 40...")
|
|
||||||
fb = subprocess.run(
|
|
||||||
["openclaw", "sessions", "compact", key, "--agent", agent, "--max-lines", "40", "--timeout", str(RPC_TIMEOUT_MS)],
|
|
||||||
capture_output=True, text=True
|
|
||||||
)
|
|
||||||
if fb.returncode == 0:
|
|
||||||
print(f" ✓ Truncation fallback success: {fb.stdout.strip()}")
|
|
||||||
else:
|
|
||||||
print(f" ✗ Truncation fallback failed: {fb.stderr.strip()}")
|
|
||||||
except Exception as e:
|
|
||||||
print(f" ✗ Exception deep-compacting {agent}: {e}")
|
|
||||||
time.sleep(2)
|
|
||||||
|
|
||||||
def get_session_metrics(agent, key, sid=None):
|
|
||||||
db_path = f"/home/node/.openclaw/agents/{agent}/agent/openclaw-agent.sqlite"
|
|
||||||
if not os.path.exists(db_path):
|
|
||||||
return 0, 0
|
|
||||||
try:
|
|
||||||
con = sqlite3.connect(db_path, timeout=5.0)
|
|
||||||
cur = con.cursor()
|
|
||||||
if not sid:
|
|
||||||
row = cur.execute("SELECT current_session_id FROM session_nodes WHERE session_key=?", (key,)).fetchone()
|
|
||||||
sid = row[0] if row else None
|
|
||||||
if not sid:
|
|
||||||
con.close()
|
|
||||||
return 0, 0
|
|
||||||
ev_cnt, ev_bytes = cur.execute(
|
|
||||||
"SELECT count(*), coalesce(sum(length(event_json)), 0) FROM transcript_events WHERE session_id=?",
|
|
||||||
(sid,)
|
|
||||||
).fetchone()
|
|
||||||
con.close()
|
|
||||||
return ev_cnt, ev_bytes
|
|
||||||
except Exception:
|
|
||||||
return 0, 0
|
|
||||||
|
|
||||||
def compact_active_sessions():
|
|
||||||
tier_a_keys = set(t[1] for t in TIER_A_TARGETS)
|
|
||||||
print(f"=== Tier B: Running Multi-Agent Compaction (threshold: {THRESHOLD_TOKENS} tokens / {THRESHOLD_TRANSCRIPT_BYTES//1024}KB, LLM summarization) ===")
|
|
||||||
res = subprocess.run(["openclaw", "sessions", "--all-agents", "--json"], capture_output=True, text=True)
|
|
||||||
if res.returncode != 0:
|
|
||||||
print(f"Failed to list sessions: {res.stderr}")
|
|
||||||
return
|
|
||||||
|
|
||||||
sessions = json.loads(res.stdout).get("sessions", [])
|
|
||||||
compacted = 0
|
|
||||||
skipped = 0
|
|
||||||
|
|
||||||
for s in sessions:
|
|
||||||
key = s.get("key", "")
|
|
||||||
agent = s.get("agentId") or "main"
|
|
||||||
tokens = s.get("totalTokens") or 0
|
|
||||||
sid = s.get("sessionId")
|
|
||||||
|
|
||||||
# Skip Tier A agents as they are handled via hard-reset
|
|
||||||
if key in tier_a_keys:
|
|
||||||
continue
|
|
||||||
|
|
||||||
# Target main sessions and all telegram sessions (topics and direct)
|
|
||||||
is_target = (
|
|
||||||
key == "agent:main:main" or
|
|
||||||
":telegram:" in key or
|
|
||||||
(key.startswith("agent:") and key.endswith(":main"))
|
|
||||||
)
|
|
||||||
if is_target and not key.endswith(":heartbeat"):
|
|
||||||
ev_cnt, ev_bytes = get_session_metrics(agent, key, sid)
|
|
||||||
needs_compaction = (
|
|
||||||
tokens >= THRESHOLD_TOKENS or
|
|
||||||
ev_bytes >= THRESHOLD_TRANSCRIPT_BYTES or
|
|
||||||
(tokens == 0 and ev_cnt >= THRESHOLD_EVENT_COUNT)
|
|
||||||
)
|
|
||||||
if needs_compaction:
|
|
||||||
stat_str = f"{tokens} tokens" if tokens else f"{ev_bytes//1024}KB / {ev_cnt} events"
|
|
||||||
print(f"Compacting {agent} ({stat_str}) -> {key}...")
|
|
||||||
c_res = subprocess.run(
|
|
||||||
["openclaw", "sessions", "compact", key, "--agent", agent, "--timeout", str(RPC_TIMEOUT_MS)],
|
|
||||||
capture_output=True, text=True
|
|
||||||
)
|
|
||||||
if c_res.returncode == 0:
|
|
||||||
print(f" ✓ {c_res.stdout.strip()}")
|
|
||||||
compacted += 1
|
|
||||||
elif "Already compacted" in c_res.stderr or "Already compacted" in c_res.stdout:
|
|
||||||
print(f" ✓ Already compacted.")
|
|
||||||
skipped += 1
|
|
||||||
else:
|
|
||||||
print(f" ✗ {c_res.stderr.strip()}; retrying with --max-lines 80 fallback...")
|
|
||||||
fb_res = subprocess.run(
|
|
||||||
["openclaw", "sessions", "compact", key, "--agent", agent, "--max-lines", "80", "--timeout", str(RPC_TIMEOUT_MS)],
|
|
||||||
capture_output=True, text=True
|
|
||||||
)
|
|
||||||
if fb_res.returncode == 0:
|
|
||||||
print(f" ✓ (fallback) {fb_res.stdout.strip()}")
|
|
||||||
compacted += 1
|
|
||||||
else:
|
|
||||||
print(f" ✗ Fallback failed: {fb_res.stderr.strip()}")
|
|
||||||
time.sleep(5) # 5s pause to avoid rotational HDD I/O saturation
|
|
||||||
else:
|
|
||||||
skipped += 1
|
|
||||||
|
|
||||||
print(f"Compaction complete: {compacted} compacted, {skipped} skipped.")
|
|
||||||
|
|
||||||
def checkpoint_wal_journals():
|
|
||||||
print("=== Checkpointing SQLite WAL journals across all agent databases ===")
|
|
||||||
dbs = ["/home/node/.openclaw/state/openclaw.sqlite"] + glob.glob("/home/node/.openclaw/agents/*/agent/openclaw-agent.sqlite")
|
|
||||||
wal_freed = 0
|
|
||||||
for db in dbs:
|
|
||||||
wal = db + "-wal"
|
|
||||||
if os.path.exists(wal):
|
|
||||||
sz = os.path.getsize(wal)
|
|
||||||
try:
|
|
||||||
con = sqlite3.connect(db, timeout=30.0)
|
|
||||||
con.execute("PRAGMA wal_checkpoint(TRUNCATE);")
|
|
||||||
con.close()
|
|
||||||
wal_freed += sz
|
|
||||||
except Exception as e:
|
|
||||||
print(f" ✗ Error checkpointing {db}: {e}")
|
|
||||||
print(f"WAL Checkpoint complete: {wal_freed / (1024*1024):.2f} MB merged to disk.")
|
|
||||||
|
|
||||||
def vacuum_and_optimize():
|
|
||||||
print("=== Running SQLite OPTIMIZE across all databases ===")
|
|
||||||
dbs = ["/home/node/.openclaw/state/openclaw.sqlite"] + glob.glob("/home/node/.openclaw/agents/*/agent/openclaw-agent.sqlite")
|
|
||||||
for db in dbs:
|
|
||||||
try:
|
|
||||||
con = sqlite3.connect(db, timeout=30.0)
|
|
||||||
con.execute("PRAGMA optimize;")
|
|
||||||
con.close()
|
|
||||||
except Exception as e:
|
|
||||||
print(f" ✗ Error optimizing {db}: {e}")
|
|
||||||
print("SQLite optimization complete.")
|
|
||||||
|
|
||||||
def main():
|
|
||||||
prewarm_page_cache()
|
|
||||||
try:
|
|
||||||
subprocess.run(["python3", "/home/node/.openclaw/scripts/recover_sessions.py"], check=False)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
prune_stale_cron_sessions()
|
|
||||||
compact_tier_a_sessions()
|
|
||||||
compact_active_sessions()
|
|
||||||
checkpoint_wal_journals()
|
|
||||||
vacuum_and_optimize()
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
EOF
|
|
||||||
chmod +x /home/node/.openclaw/scripts/compact_sessions.py
|
|
||||||
'
|
|
||||||
|
|
||||||
# 2. Register / Update the OpenClaw cron job
|
|
||||||
echo "Checking existing OpenClaw Cron automation '$JOB_NAME'..."
|
|
||||||
EXISTING_ID=$(kubectl exec -n "$NAMESPACE" "$POD_NAME" -c "$CONTAINER_NAME" -- openclaw cron list --json 2>/dev/null | python3 -c 'import sys, json; data=json.load(sys.stdin); print(next((j["id"] for j in data.get("jobs", []) if j.get("declarationKey") == "'"$JOB_NAME"'"), ""))' 2>/dev/null || true)
|
|
||||||
|
|
||||||
if [ -n "$EXISTING_ID" ]; then
|
|
||||||
echo "Updating existing OpenClaw Cron automation '$EXISTING_ID' ($CRON_SCHEDULE @ $TIMEZONE)..."
|
|
||||||
kubectl exec -n "$NAMESPACE" "$POD_NAME" -c "$CONTAINER_NAME" -- openclaw cron edit "$EXISTING_ID" \
|
|
||||||
--cron "$CRON_SCHEDULE" \
|
--cron "$CRON_SCHEDULE" \
|
||||||
--tz "$TIMEZONE" \
|
--tz "$TIMEZONE" \
|
||||||
--command "python3 /home/node/.openclaw/scripts/compact_sessions.py"
|
--command "python3 /home/node/.openclaw/scripts/compact_sessions.py"
|
||||||
else
|
else
|
||||||
echo "Registering new OpenClaw Cron automation '$JOB_NAME' ($CRON_SCHEDULE @ $TIMEZONE)..."
|
echo "Registering new nightly compaction automation '$NIGHTLY_JOB' ($CRON_SCHEDULE @ $TIMEZONE)..."
|
||||||
kubectl exec -n "$NAMESPACE" "$POD_NAME" -c "$CONTAINER_NAME" -- openclaw cron add \
|
kubectl exec -n "$NAMESPACE" "$POD_NAME" -c "$CONTAINER_NAME" -- openclaw cron add \
|
||||||
--name "Nightly Session Compaction" \
|
--name "Nightly Session Compaction" \
|
||||||
--declaration-key "$JOB_NAME" \
|
--declaration-key "$NIGHTLY_JOB" \
|
||||||
--description "Nightly multi-agent session compaction and SQLite WAL checkpointing" \
|
--description "Nightly multi-agent session compaction and SQLite WAL checkpointing" \
|
||||||
--cron "$CRON_SCHEDULE" \
|
--cron "$CRON_SCHEDULE" \
|
||||||
--tz "$TIMEZONE" \
|
--tz "$TIMEZONE" \
|
||||||
@@ -344,7 +82,4 @@ else
|
|||||||
--command "python3 /home/node/.openclaw/scripts/compact_sessions.py"
|
--command "python3 /home/node/.openclaw/scripts/compact_sessions.py"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "=== Compaction Cron automation successfully registered/updated! ==="
|
echo "=== All session maintenance automations successfully registered/updated! ==="
|
||||||
if [ -n "$EXISTING_ID" ]; then
|
|
||||||
kubectl exec -n "$NAMESPACE" "$POD_NAME" -c "$CONTAINER_NAME" -- openclaw cron get "$EXISTING_ID" 2>/dev/null || true
|
|
||||||
fi
|
|
||||||
|
|||||||
Reference in New Issue
Block a user