infrapuzzle/k8s/monitoring/README.md

90 lines
2.3 KiB
Markdown

# Grafana Dashboards as Code
This directory contains Grafana dashboards managed as Kubernetes ConfigMaps. Dashboards are automatically loaded into Grafana via the dashboard sidecar.
## Architecture
```
dashboards/*.yaml (ConfigMap with embedded JSON)
│ kubectl apply
Kubernetes ConfigMap (label: grafana_dashboard=1)
│ Sidecar watches for ConfigMaps
Grafana Dashboard (read-only, GitOps managed)
```
## Quick Deploy
```bash
# Apply all dashboards
kubectl apply -f dashboards/
# Apply a single dashboard
kubectl apply -f dashboards/grafana-dashboard-home-climate.yaml
```
## Verify
```bash
# Check ConfigMaps are created and labeled correctly
kubectl get configmaps -n monitoring -l grafana_dashboard=1
# Check sidecar logs to confirm dashboards were loaded
kubectl logs -n monitoring deployment/kube-prometheus-stack-grafana -c grafana-sc-dashboards --tail=50
# List dashboards via Grafana API
kubectl -n monitoring exec deploy/kube-prometheus-stack-grafana -- \
curl -s -u admin:$(kubectl -n monitoring get secret kube-prometheus-stack-grafana -o jsonpath='{.data.admin-password}' | base64 -d) \
http://localhost:3000/api/search?type=dash-db | jq '.[].title'
```
## Discovery
Before generating new dashboards, discover what metrics are available:
```bash
# Discover InfluxDB measurements, fields, and tags
./scripts/discover-influxdb.sh
# Discover Home Assistant entities and their metadata
./scripts/discover-home-assistant.sh
```
## Dashboard Structure
Each dashboard is a single YAML file containing a Kubernetes ConfigMap with the dashboard JSON embedded:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: grafana-dashboard-<name>
namespace: monitoring
labels:
grafana_dashboard: "1"
annotations:
grafana_dashboard_folder: "<Folder Name>"
data:
<name>.json: |
{
"dashboard": { ... }
}
```
## Adding a New Dashboard
1. Run discovery scripts to understand available metrics
2. Create a new ConfigMap YAML in `dashboards/`
3. Apply: `kubectl apply -f dashboards/grafana-dashboard-<name>.yaml`
4. Verify the dashboard appears in Grafana at https://grafana.haumdaucher.de
## Removing a Dashboard
```bash
kubectl delete configmap -n monitoring grafana-dashboard-<name>
```
The sidecar will automatically remove the dashboard from Grafana within ~60 seconds.