monitoring: optimize alerts and mitigate fritzbox-exporter timeouts

2026-07-11 07:40:06 +02:00
parent c5458821a7
commit bdabc56afe
10 changed files with 269 additions and 14 deletions
@@ -160,3 +160,53 @@ spec:
 *   **Namespaces**: Every application gets its own namespace.
 *   **Secrets**: Encrypt all secrets using `git-crypt`.

+## Network Setup (VPN & Routing)
+
+The Kubernetes cluster (Haumdaucher) is connected to the home network via a dedicated WireGuard VPN tunnel.
+
+```mermaid
+graph TD
+    subgraph "Kubernetes Cluster (haumdaucher.de - 136.243.23.215)"
+        subgraph "monitoring namespace"
+            Exporter[fritzbox-exporter Pod]
+        end
+        HostNet[Host Network Namespace]
+        WgPod[wireguard Pod hostNetwork: true]
+    end
+
+    subgraph "Home Network (192.168.10.0/24)"
+        FB[FRITZ!Box Gateway - 192.168.10.1]
+        Taupi[Taupi Fan Shelly - 192.168.10.168]
+    end
+
+    Exporter -- "Queries 192.168.10.1" --> HostNet
+    HostNet -- "Route: 192.168.10.0/24 via wg0" --> WgPod
+    WgPod -- "WireGuard VPN Tunnel (51820/UDP)" --> FB
+    FB -- "Accesses local subnet" --> Taupi
+```
+
+### Components & Routing
+1. **Home Network**: `192.168.10.0/24`. Contains the home devices (e.g., Shelly plug at `192.168.10.168`) and the gateway FRITZ!Box at `192.168.10.1`.
+2. **Kubernetes Cluster Network**: Pod subnets (`10.233.64.0/24` etc.) running on the remote public server (`136.243.23.215`).
+3. **WireGuard VPN Link (`wg0`)**:
+   - The `wireguard` pod in the `wireguard` namespace is configured with `hostNetwork: true`. This exposes the `wg0` interface directly in the host's root network namespace.
+   - The cluster host has a static tunnel IP of `192.168.11.1/24` on `wg0`.
+   - The FRITZ!Box acts as the active peer client, initiating the connection to the host node at `136.243.23.215:51820`.
+   - The host routing table directs home network traffic over the tunnel:
+     ```bash
+     192.168.10.0/24 dev wg0 scope link
+     ```
+   - Pods inside the cluster (like `fritzbox-exporter`) query `192.168.10.1`. The traffic is forwarded by the host's default CNI routing to the host's network namespace, which matching the `192.168.10.0/24` subnet route and sends it over `wg0` to the FRITZ!Box.
+
+### FritzBox Exporter Egress Reject Workaround (Option B)
+
+**The Problem**:
+When the `fritzbox-exporter` pod queries the FRITZ!Box TR-064 API at `192.168.10.1` from outside the home LAN subnet (using the transit WireGuard subnet IP `192.168.11.1`), the FRITZ!Box's internal security policy redirects the client to its public WAN IP (e.g. `212.42.244.122`) for portal login/authentication (`login_sid.lua`). 
+When the dynamic WAN IP of the FRITZ!Box changes or the WAN ports are closed, these queries to the public IP timeout. Because Go's default HTTP client doesn't enforce a timeout, the exporter hangs for Go/Linux's default TCP connection handshake timeout of **2 minutes (120 seconds)**. This blocks all other metric scrapes and causes Prometheus target scrape timeouts.
+
+**The Solution**:
+Rather than disabling CPU/RAM/temperature metrics (`-nolua`) or modifying the remote FRITZ!Box WireGuard subnet configuration, we block the exporter pod from reaching the public WAN IP.
+- We add an `initContainer` in `fritzbox-exporter.yaml` running with `NET_ADMIN` privileges.
+- The `initContainer` installs `iptables` and adds rules directly to the pod's shared network namespace.
+- It permits traffic to local private subnets (RFC1918 ranges) but rejects all TCP egress to public WAN IPs on port 80/443 with a `TCP RST` (Reset) immediately.
+- This causes redirected authentication requests to fail in milliseconds instead of hanging for 2 minutes. The exporter immediately falls back to scrape TR-064 metrics (WAN sync speed, bytes) successfully, avoiding Prometheus scrapes timeouts.