infrapuzzle/k8s/openclaw

OpenClaw Operations Guide (Operator-Managed)

This directory manages the deployment of the OpenClaw stateful AI agent on the single-node haumdaucher cluster.

The deployment is managed by the official OpenClaw Operator (openclaw-operator) using an OpenClawInstance Custom Resource, replacing the legacy 571-line manual deployment with a modern, zero-drift GitOps-friendly framework.

---

🏗️ Architecture Layout

• Operator System: Watches the openclaw namespace and reconciles the OpenClawInstance resource.
• Deep-Merge Config: Configured with mergeMode: merge to prevent configuration clobbering. Declarative settings from Git are safely merged with dynamic configurations (such as linked Telegram accounts or runtime credentials) mutated by the agent on the PVC.
• Built-in Sidecars:
  • Headless Chromium: Auto-injected and wired to http://127.0.0.1:9222 for browser automation and tools.
  • Gateway Proxy: Handles token-based authentication and secure WebSocket forwarding natively.
• Custom Sidecars:
  • git-sync: Pulls custom skills dynamically from your private repository git.moritzgraf.de/moritz/mop-skills.
  • skill-stabilizer: Continuously copies checked-out skills into the flat /home/openclaw/.openclaw/skills workspace.
• Memory & Knowledge Wiki (Karpathy LLM Wiki):
  • Vector Search: Semantic memory search via OpenRouter /v1/embeddings (text-embedding-3-small).
  • Memory Wiki: Bundled memory-wiki compiles structured entities, concepts, and dashboards into an Obsidian-compatible vault (wiki/main).
  • Automated Backup: openclaw-wiki-backup CronJob runs daily at 03:30 AM UTC to commit and push the knowledge vault to git.moritzgraf.de/moritz/knowledge-wiki.
• Multi-Agent Specialist Delegation (ADR 0001):
  • Conductor + 9 Subagents: Lean front-of-house coordinator orchestrates 9 specialized domain subagents defined as code in the knowledge wiki (agents/*.md).
  • Direct Fast Path (< 2s): Simple status, home automation toggles, and single calendar entries run directly in-place.
  • Domain Stickiness: Multi-turn dialogue (e.g. step-by-step cooking or vacation planning) stays pinned to the active subagent with persistent session continuity.
  • Active MCP Servers: stadtrat (ALLRIS Net), supermarket (Marktguru), geizhals (price histories), ebay, garmin (148 health tools), bring (4 lists), spotify, calendar (family + city), jelly (Jellyfin/Jellyseerr), homeassistant, prometheus, gitea, weather.

---

🚀 Installation & Setup

1. Install the Operator (One-time)

The operator is installed globally via Helm:

helm install openclaw-operator \
  oci://ghcr.io/openclaw-rocks/charts/openclaw-operator \
  --namespace openclaw-operator-system \
  --create-namespace

2. Apply Custom Secrets and CRD Instance

The deployment resources are applied directly via kubectl:

# Apply raw secrets (git-crypt encrypted in the repository)
kubectl apply -f k8s/openclaw/openclaw.secret.yaml

# Apply the custom resource definition instance
kubectl apply -f k8s/openclaw/openclaw-instance.secret.yaml

# Apply the automated daily knowledge-wiki backup CronJob
kubectl apply -f k8s/openclaw/cronjob-wiki-backup.secret.yaml

# Register the IT Guy weekly health audit automation
./k8s/openclaw/setup-itguy-cron.sh

---

🔑 Gateway Access (Token Auth)

Basic Auth (htpasswd) has been retired. The operator-injected gateway proxy implements token-based authentication.

To access the Control UI in your browser, append your gateway token as a URL fragment:

https://openclaw.haumdaucher.de/#token=<your-gateway-token>

The <your-gateway-token> corresponds to the gateway-token value configured inside the openclaw-secrets Secret.

---

💾 Backup & Disaster Recovery

Because your deployment relies on openebs-hostpath persistent storage, all state resides on a single physical node disk. Local pre-migration backups should be kept in case of host or cluster recreation.

Perform a Local Backup

Run the following script to pull your memory files, SQLite databases, paired devices, and active configs locally:

# Define target backup dir
BACKUP_DIR="k8s/openclaw/backup"
mkdir -p "$BACKUP_DIR"

# Get active pod name
POD_NAME=$(kubectl get pods -n openclaw -l app.kubernetes.io/instance=openclaw -o jsonpath='{.items[0].metadata.name}')

# Copy critical runtime files (excluding 2.9GB model files)
for item in openclaw.json workspace memory credentials identity devices telegram settings scripts tasks subagents canvas; do
  kubectl cp -n openclaw -c openclaw "$POD_NAME:/home/openclaw/.openclaw/$item" "$BACKUP_DIR/$item"
done

Restore from a Local Backup

If your PVC is recreated or wiped, you can push the backup files back to the new pod:

# Push directories back to the running pod
kubectl cp "$BACKUP_DIR/workspace" openclaw-0:/home/openclaw/.openclaw/workspace -n openclaw -c openclaw
# Repeat for other folders as needed

---

🔧 Operational Verification Commands

# Check the controller reconciliation phase (Expected: PHASE=Running)
kubectl get openclawinstances -n openclaw

# View the status of all 6 containers inside the StatefulSet pod
kubectl get pods -n openclaw -o wide

# Review live logs of the core OpenClaw engine
kubectl logs -n openclaw openclaw-0 -c openclaw --tail=100 -f

# Review live logs of the Chromium sidecar
kubectl logs -n openclaw openclaw-0 -c chromium --tail=100 -f