moritz
/
infrapuzzle
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
infrapuzzle/k8s/monitoring/DASHBOARD_CREATION.md

8.7 KiB
Raw Blame History

DASHBOARD_CREATION.md — How to Create & Update Grafana Dashboards

Linked from: AGENTS.md

This document captures the project-specific process for creating Grafana dashboards in this folder. For general Grafana dashboard JSON schema, panel types, units, transformations, and API workflows, use the Grafana skills (dashboarding, grafana-oss, promql) — they are the authoritative reference for Grafana-native topics.

[!IMPORTANT] Always read the skills first before creating a dashboard:

  • dashboarding — JSON schema, panel types, units, template variables, transformations, links, annotations
  • grafana-oss — Dashboard provisioning, datasource config, RBAC, plugin provisioning
  • promql — PromQL query patterns, rate/irate, histograms, recording rules

Step 1: Identify the Data Source

This project has two types of dashboard data sources:

Data Source Type When to Use Query Language
Prometheus prometheus Kubernetes metrics, custom exporters (taupi-fan, tankerkoenig) PromQL
InfluxDB Home Assistant influxdb (Flux) Home Assistant sensor data (temperature, humidity, energy, etc.) Flux

Prometheus dashboards are simpler — just write PromQL expressions in targets[].expr. See the promql skill and existing examples like grafana-dashboard-taupi-fan.yaml.

InfluxDB dashboards require Flux queries with project-specific conventions (see § InfluxDB Flux Queries below).

Datasource UIDs

Datasource references in panels require the actual UID, not the display name. UIDs can change when Grafana is redeployed. Current known UIDs are documented in AGENTS.md — always verify before creating a new dashboard:

kubectl -n monitoring exec deploy/prometheus-operator-grafana -c grafana -- \
  curl -s -u "admin:<GRAFANA_PASSWORD>" \
  http://localhost:3000/api/datasources | jq '.[] | {name, uid, type}'

The Grafana admin password is in prometheus-operator.secret.yml under grafana.adminPassword.

Step 2: Discover Available Metrics

For Prometheus Dashboards

Query Prometheus directly or use the Grafana Explore UI:

# Port-forward to Prometheus
kubectl -n monitoring port-forward svc/prometheus-operated 9090

# Browse available metrics
curl -s http://localhost:9090/api/v1/label/__name__/values | jq '.data[]' | grep '<prefix>'

For InfluxDB Dashboards (Home Assistant Data)

Use the helper scripts to discover what data exists:

# Query Home Assistant for entity metadata
./scripts/discover-home-assistant.sh

# Query InfluxDB for measurements and schema
# (requires port-forward first: kubectl -n influxdb port-forward svc/influxdb-influxdb2 8086:80)
./scripts/discover-influxdb.sh

Sensor selection tip: Use Wandthermostat entities for temperature/humidity (they provide calibrated readings). Filter by wandthermostat or thermostat in the entity_id.

Step 3: Create the Dashboard ConfigMap

Dashboards are deployed as Kubernetes ConfigMaps with the label grafana_dashboard: "1". The Grafana sidecar (grafana-sc-dashboard container) watches for these and auto-loads them.

ConfigMap Template

apiVersion: v1
kind: ConfigMap
metadata:
  name: grafana-dashboard-<name>
  namespace: monitoring
  labels:
    grafana_dashboard: "1"
  annotations:
    grafana_dashboard_folder: "🏠 Home"
data:
  <name>.json: |
    {
      "uid": "<unique-uid>",
      "title": "<Dashboard Title>",
      "tags": ["<category>"],
      "timezone": "browser",
      "schemaVersion": 39,
      "version": 1,
      "refresh": "30s",
      "editable": false,
      "graphTooltip": 2,
      "time": { "from": "now-24h", "to": "now" },
      "panels": [ ... ]
    }

[!CAUTION] No "dashboard" wrapper. The JSON must have uid, title, panels at the top level. The {"dashboard": {...}} wrapper is only for the Grafana HTTP API — file provisioning via the sidecar will fail with "Dashboard title cannot be empty" if you use it.

For the full panel JSON schema, units, thresholds, overrides, and transformations, see the dashboarding skill's references/json-schema.md.

Project-Specific Dashboard Conventions

  • "editable": false — strict GitOps; never edit in the Grafana UI
  • "graphTooltip": 2 — shared crosshair + tooltip across all panels
  • Use emoji in titles for visual appeal (🏠 ⚡ 🔥 🌡️ 💧 💨)
  • Include calcs in legend: ["lastNotNull", "min", "max", "mean"]
  • Use "spanNulls": true to handle gaps in sensor data
  • Use semantic threshold colors: blue=cold, green=comfort, orange=warm, red=hot

Step 4: Deploy & Verify

# Apply
kubectl apply -f dashboards/grafana-dashboard-<name>.yaml

# Verify ConfigMap labeling
kubectl get configmaps -n monitoring -l grafana_dashboard=1

# Check sidecar pickup (container name is singular: grafana-sc-dashboard)
kubectl -n monitoring logs -l "app.kubernetes.io/name=grafana" \
  -c grafana-sc-dashboard --tail=20

# Check for Grafana errors
kubectl -n monitoring logs -l "app.kubernetes.io/name=grafana" \
  -c grafana --tail=30 | grep -i "error\|failed"

Updating an Existing Dashboard

  1. Edit the YAML file in dashboards/
  2. Increment the version field in the dashboard JSON
  3. kubectl apply -f dashboards/grafana-dashboard-<name>.yaml
  4. Sidecar detects the change within ~60 seconds

Removing a Dashboard

kubectl delete configmap -n monitoring grafana-dashboard-<name>

InfluxDB Flux Queries

This section covers the project-specific Flux conventions for querying Home Assistant data in InfluxDB. These patterns are unique to this project and not covered by the Grafana skills.

Data Schema

Home Assistant writes to InfluxDB with this structure:

  • Measurement name = unit of measurement (e.g., °C, %, W, kWh)
  • Tags: entity_id (e.g., home_wohnzimmer_temperature), domain (e.g., sensor)
  • Fields: value (numeric), plus metadata strings (friendly_name_str, etc.)

Critical Flux Query Rules

[!WARNING] These rules come from hard-won debugging. Violating any of them will result in "no data" panels.

  1. entity_id has NO sensor. prefix — HA stores the domain in a separate domain tag. Use home_wohnzimmer_temperature, NOT sensor.home_wohnzimmer_temperature.
  2. Always use bracket notation for underscore-prefixed fields: r["_measurement"], r["_field"] — dot notation (r._measurement) may silently fail.
  3. Always use Grafana time variables — v.timeRangeStart, v.timeRangeStop, v.windowPeriod. Never hardcode range(start: -24h).
  4. Always end with |> yield(name: "mean") — required for Grafana to parse Flux results.
  5. Always filter r["_field"] == "value" — InfluxDB stores metadata strings in other fields.
  6. Use regex for multi-entity panels — r.entity_id =~ /^home_/ instead of or chains (Flux or inside a single filter is unreliable).

Flux Query Templates

Time series panel (aggregated):

from(bucket: "default")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "<UNIT>" and r["_field"] == "value")
  |> filter(fn: (r) => r.entity_id == "<ENTITY_ID>")
  |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false)
  |> yield(name: "mean")

Stat panel (current value):

from(bucket: "default")
  |> range(start: -15m)
  |> filter(fn: (r) => r["_measurement"] == "<UNIT>" and r["_field"] == "value")
  |> filter(fn: (r) => r.entity_id == "<ENTITY_ID>")
  |> last()
  |> yield(name: "mean")

Multi-entity panel (regex):

from(bucket: "default")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "<UNIT>" and r["_field"] == "value")
  |> filter(fn: (r) => r.entity_id =~ /^home_/)
  |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false)
  |> yield(name: "mean")

InfluxDB Measurement Reference

Measurement Grafana Unit Typical Entities
°C celsius Wandthermostat temperatures, outdoor temp, battery temp
% percent Humidity, battery levels, heating valve position, cloud cover
W watt Solar production, self-consumption, grid import/export
kWh kwatth Daily energy totals (production, consumption, import, export)
m³ m3 Gas volume counter
m³/h m3/h Current gas flow rate
hPa pressurehpa Barometric pressure
km/h velocitykmh Wind speed
mm/h lengthmm Precipitation rate
EUR currencyEUR Gas cost, electricity cost