227 lines
8.7 KiB
Markdown
227 lines
8.7 KiB
Markdown
# DASHBOARD_CREATION.md — How to Create & Update Grafana Dashboards
|
|
|
|
> **Linked from**: [AGENTS.md](AGENTS.md)
|
|
|
|
This document captures the project-specific process for creating Grafana dashboards in this folder. For general Grafana dashboard JSON schema, panel types, units, transformations, and API workflows, use the **Grafana skills** (`dashboarding`, `grafana-oss`, `promql`) — they are the authoritative reference for Grafana-native topics.
|
|
|
|
> [!IMPORTANT]
|
|
> **Always read the skills first** before creating a dashboard:
|
|
> - **`dashboarding`** — JSON schema, panel types, units, template variables, transformations, links, annotations
|
|
> - **`grafana-oss`** — Dashboard provisioning, datasource config, RBAC, plugin provisioning
|
|
> - **`promql`** — PromQL query patterns, rate/irate, histograms, recording rules
|
|
|
|
---
|
|
|
|
## Step 1: Identify the Data Source
|
|
|
|
This project has **two** types of dashboard data sources:
|
|
|
|
| Data Source | Type | When to Use | Query Language |
|
|
|---|---|---|---|
|
|
| Prometheus | `prometheus` | Kubernetes metrics, custom exporters (taupi-fan, tankerkoenig) | PromQL |
|
|
| InfluxDB Home Assistant | `influxdb` (Flux) | Home Assistant sensor data (temperature, humidity, energy, etc.) | Flux |
|
|
|
|
**Prometheus dashboards** are simpler — just write PromQL expressions in `targets[].expr`. See the `promql` skill and existing examples like [grafana-dashboard-taupi-fan.yaml](dashboards/grafana-dashboard-taupi-fan.yaml).
|
|
|
|
**InfluxDB dashboards** require Flux queries with project-specific conventions (see [§ InfluxDB Flux Queries](#influxdb-flux-queries) below).
|
|
|
|
### Datasource UIDs
|
|
|
|
Datasource references in panels require the **actual UID**, not the display name. UIDs can change when Grafana is redeployed. Current known UIDs are documented in [AGENTS.md](AGENTS.md) — always verify before creating a new dashboard:
|
|
|
|
```bash
|
|
kubectl -n monitoring exec deploy/prometheus-operator-grafana -c grafana -- \
|
|
curl -s -u "admin:<GRAFANA_PASSWORD>" \
|
|
http://localhost:3000/api/datasources | jq '.[] | {name, uid, type}'
|
|
```
|
|
|
|
The Grafana admin password is in [`prometheus-operator.secret.yml`](prometheus-operator.secret.yml) under `grafana.adminPassword`.
|
|
|
|
---
|
|
|
|
## Step 2: Discover Available Metrics
|
|
|
|
### For Prometheus Dashboards
|
|
|
|
Query Prometheus directly or use the Grafana Explore UI:
|
|
|
|
```bash
|
|
# Port-forward to Prometheus
|
|
kubectl -n monitoring port-forward svc/prometheus-operated 9090
|
|
|
|
# Browse available metrics
|
|
curl -s http://localhost:9090/api/v1/label/__name__/values | jq '.data[]' | grep '<prefix>'
|
|
```
|
|
|
|
### For InfluxDB Dashboards (Home Assistant Data)
|
|
|
|
Use the helper scripts to discover what data exists:
|
|
|
|
```bash
|
|
# Query Home Assistant for entity metadata
|
|
./scripts/discover-home-assistant.sh
|
|
|
|
# Query InfluxDB for measurements and schema
|
|
# (requires port-forward first: kubectl -n influxdb port-forward svc/influxdb-influxdb2 8086:80)
|
|
./scripts/discover-influxdb.sh
|
|
```
|
|
|
|
**Sensor selection tip**: Use **Wandthermostat** entities for temperature/humidity (they provide calibrated readings). Filter by `wandthermostat` or `thermostat` in the entity_id.
|
|
|
|
---
|
|
|
|
## Step 3: Create the Dashboard ConfigMap
|
|
|
|
Dashboards are deployed as **Kubernetes ConfigMaps** with the label `grafana_dashboard: "1"`. The Grafana sidecar (`grafana-sc-dashboard` container) watches for these and auto-loads them.
|
|
|
|
### ConfigMap Template
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: ConfigMap
|
|
metadata:
|
|
name: grafana-dashboard-<name>
|
|
namespace: monitoring
|
|
labels:
|
|
grafana_dashboard: "1"
|
|
annotations:
|
|
grafana_dashboard_folder: "🏠 Home"
|
|
data:
|
|
<name>.json: |
|
|
{
|
|
"uid": "<unique-uid>",
|
|
"title": "<Dashboard Title>",
|
|
"tags": ["<category>"],
|
|
"timezone": "browser",
|
|
"schemaVersion": 39,
|
|
"version": 1,
|
|
"refresh": "30s",
|
|
"editable": false,
|
|
"graphTooltip": 2,
|
|
"time": { "from": "now-24h", "to": "now" },
|
|
"panels": [ ... ]
|
|
}
|
|
```
|
|
|
|
> [!CAUTION]
|
|
> **No `"dashboard"` wrapper.** The JSON must have `uid`, `title`, `panels` at the top level. The `{"dashboard": {...}}` wrapper is only for the Grafana HTTP API — file provisioning via the sidecar will fail with *"Dashboard title cannot be empty"* if you use it.
|
|
|
|
For the full panel JSON schema, units, thresholds, overrides, and transformations, see the **`dashboarding`** skill's `references/json-schema.md`.
|
|
|
|
### Project-Specific Dashboard Conventions
|
|
|
|
- `"editable": false` — strict GitOps; never edit in the Grafana UI
|
|
- `"graphTooltip": 2` — shared crosshair + tooltip across all panels
|
|
- Use emoji in titles for visual appeal (🏠 ⚡ 🔥 🌡️ 💧 💨)
|
|
- Include `calcs` in legend: `["lastNotNull", "min", "max", "mean"]`
|
|
- Use `"spanNulls": true` to handle gaps in sensor data
|
|
- Use semantic threshold colors: blue=cold, green=comfort, orange=warm, red=hot
|
|
|
|
---
|
|
|
|
## Step 4: Deploy & Verify
|
|
|
|
```bash
|
|
# Apply
|
|
kubectl apply -f dashboards/grafana-dashboard-<name>.yaml
|
|
|
|
# Verify ConfigMap labeling
|
|
kubectl get configmaps -n monitoring -l grafana_dashboard=1
|
|
|
|
# Check sidecar pickup (container name is singular: grafana-sc-dashboard)
|
|
kubectl -n monitoring logs -l "app.kubernetes.io/name=grafana" \
|
|
-c grafana-sc-dashboard --tail=20
|
|
|
|
# Check for Grafana errors
|
|
kubectl -n monitoring logs -l "app.kubernetes.io/name=grafana" \
|
|
-c grafana --tail=30 | grep -i "error\|failed"
|
|
```
|
|
|
|
### Updating an Existing Dashboard
|
|
|
|
1. Edit the YAML file in `dashboards/`
|
|
2. Increment the `version` field in the dashboard JSON
|
|
3. `kubectl apply -f dashboards/grafana-dashboard-<name>.yaml`
|
|
4. Sidecar detects the change within ~60 seconds
|
|
|
|
### Removing a Dashboard
|
|
|
|
```bash
|
|
kubectl delete configmap -n monitoring grafana-dashboard-<name>
|
|
```
|
|
|
|
---
|
|
|
|
## InfluxDB Flux Queries
|
|
|
|
This section covers the **project-specific Flux conventions** for querying Home Assistant data in InfluxDB. These patterns are unique to this project and not covered by the Grafana skills.
|
|
|
|
### Data Schema
|
|
|
|
Home Assistant writes to InfluxDB with this structure:
|
|
|
|
- **Measurement name** = unit of measurement (e.g., `°C`, `%`, `W`, `kWh`)
|
|
- **Tags**: `entity_id` (e.g., `home_wohnzimmer_temperature`), `domain` (e.g., `sensor`)
|
|
- **Fields**: `value` (numeric), plus metadata strings (`friendly_name_str`, etc.)
|
|
|
|
### Critical Flux Query Rules
|
|
|
|
> [!WARNING]
|
|
> These rules come from hard-won debugging. Violating any of them will result in "no data" panels.
|
|
|
|
1. **`entity_id` has NO `sensor.` prefix** — HA stores the domain in a separate `domain` tag. Use `home_wohnzimmer_temperature`, NOT `sensor.home_wohnzimmer_temperature`.
|
|
2. **Always use bracket notation** for underscore-prefixed fields: `r["_measurement"]`, `r["_field"]` — dot notation (`r._measurement`) may silently fail.
|
|
3. **Always use Grafana time variables** — `v.timeRangeStart`, `v.timeRangeStop`, `v.windowPeriod`. Never hardcode `range(start: -24h)`.
|
|
4. **Always end with `|> yield(name: "mean")`** — required for Grafana to parse Flux results.
|
|
5. **Always filter `r["_field"] == "value"`** — InfluxDB stores metadata strings in other fields.
|
|
6. **Use regex for multi-entity panels** — `r.entity_id =~ /^home_/` instead of `or` chains (Flux `or` inside a single filter is unreliable).
|
|
|
|
### Flux Query Templates
|
|
|
|
**Time series panel (aggregated):**
|
|
```flux
|
|
from(bucket: "default")
|
|
|> range(start: v.timeRangeStart, stop: v.timeRangeStop)
|
|
|> filter(fn: (r) => r["_measurement"] == "<UNIT>" and r["_field"] == "value")
|
|
|> filter(fn: (r) => r.entity_id == "<ENTITY_ID>")
|
|
|> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false)
|
|
|> yield(name: "mean")
|
|
```
|
|
|
|
**Stat panel (current value):**
|
|
```flux
|
|
from(bucket: "default")
|
|
|> range(start: -15m)
|
|
|> filter(fn: (r) => r["_measurement"] == "<UNIT>" and r["_field"] == "value")
|
|
|> filter(fn: (r) => r.entity_id == "<ENTITY_ID>")
|
|
|> last()
|
|
|> yield(name: "mean")
|
|
```
|
|
|
|
**Multi-entity panel (regex):**
|
|
```flux
|
|
from(bucket: "default")
|
|
|> range(start: v.timeRangeStart, stop: v.timeRangeStop)
|
|
|> filter(fn: (r) => r["_measurement"] == "<UNIT>" and r["_field"] == "value")
|
|
|> filter(fn: (r) => r.entity_id =~ /^home_/)
|
|
|> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false)
|
|
|> yield(name: "mean")
|
|
```
|
|
|
|
---
|
|
|
|
## InfluxDB Measurement Reference
|
|
|
|
| Measurement | Grafana Unit | Typical Entities |
|
|
|---|---|---|
|
|
| `°C` | `celsius` | Wandthermostat temperatures, outdoor temp, battery temp |
|
|
| `%` | `percent` | Humidity, battery levels, heating valve position, cloud cover |
|
|
| `W` | `watt` | Solar production, self-consumption, grid import/export |
|
|
| `kWh` | `kwatth` | Daily energy totals (production, consumption, import, export) |
|
|
| `m³` | `m3` | Gas volume counter |
|
|
| `m³/h` | `m3/h` | Current gas flow rate |
|
|
| `hPa` | `pressurehpa` | Barometric pressure |
|
|
| `km/h` | `velocitykmh` | Wind speed |
|
|
| `mm/h` | `lengthmm` | Precipitation rate |
|
|
| `EUR` | `currencyEUR` | Gas cost, electricity cost |
|