# DASHBOARD_CREATION.md — How to Create & Update Grafana Dashboards

> **Linked from**: [AGENTS.md](AGENTS.md)

This document captures the project-specific process for creating Grafana dashboards in this folder. For general Grafana dashboard JSON schema, panel types, units, transformations, and API workflows, use the **Grafana skills** (`dashboarding`, `grafana-oss`, `promql`) — they are the authoritative reference for Grafana-native topics.

> [!IMPORTANT]
> **Always read the skills first** before creating a dashboard:
> - **`dashboarding`** — JSON schema, panel types, units, template variables, transformations, links, annotations
> - **`grafana-oss`** — Dashboard provisioning, datasource config, RBAC, plugin provisioning
> - **`promql`** — PromQL query patterns, rate/irate, histograms, recording rules

---

## Step 1: Identify the Data Source

This project has **two** types of dashboard data sources:

| Data Source | Type | When to Use | Query Language |
|---|---|---|---|
| Prometheus | `prometheus` | Kubernetes metrics, custom exporters (taupi-fan, tankerkoenig) | PromQL |
| InfluxDB Home Assistant | `influxdb` (Flux) | Home Assistant sensor data (temperature, humidity, energy, etc.) | Flux |

**Prometheus dashboards** are simpler — just write PromQL expressions in `targets[].expr`. See the `promql` skill and existing examples like [grafana-dashboard-taupi-fan.yaml](dashboards/grafana-dashboard-taupi-fan.yaml).

**InfluxDB dashboards** require Flux queries with project-specific conventions (see [§ InfluxDB Flux Queries](#influxdb-flux-queries) below).

### Datasource UIDs

Datasource references in panels require the **actual UID**, not the display name. UIDs can change when Grafana is redeployed. Current known UIDs are documented in [AGENTS.md](AGENTS.md) — always verify before creating a new dashboard:

```bash
kubectl -n monitoring exec deploy/prometheus-operator-grafana -c grafana -- \
  curl -s -u "admin:<GRAFANA_PASSWORD>" \
  http://localhost:3000/api/datasources | jq '.[] | {name, uid, type}'
```

The Grafana admin password is in [`prometheus-operator.secret.yml`](prometheus-operator.secret.yml) under `grafana.adminPassword`.

---

## Step 2: Discover Available Metrics

### For Prometheus Dashboards

Query Prometheus directly or use the Grafana Explore UI:

```bash
# Port-forward to Prometheus
kubectl -n monitoring port-forward svc/prometheus-operated 9090

# Browse available metrics
curl -s http://localhost:9090/api/v1/label/__name__/values | jq '.data[]' | grep '<prefix>'
```

### For InfluxDB Dashboards (Home Assistant Data)

Use the helper scripts to discover what data exists:

```bash
# Query Home Assistant for entity metadata
./scripts/discover-home-assistant.sh

# Query InfluxDB for measurements and schema
# (requires port-forward first: kubectl -n influxdb port-forward svc/influxdb-influxdb2 8086:80)
./scripts/discover-influxdb.sh
```

**Sensor selection tip**: Use **Wandthermostat** entities for temperature/humidity (they provide calibrated readings). Filter by `wandthermostat` or `thermostat` in the entity_id.

---

## Step 3: Create the Dashboard ConfigMap

Dashboards are deployed as **Kubernetes ConfigMaps** with the label `grafana_dashboard: "1"`. The Grafana sidecar (`grafana-sc-dashboard` container) watches for these and auto-loads them.

### ConfigMap Template

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: grafana-dashboard-<name>
  namespace: monitoring
  labels:
    grafana_dashboard: "1"
  annotations:
    grafana_dashboard_folder: "🏠 Home"
data:
  <name>.json: |
    {
      "uid": "<unique-uid>",
      "title": "<Dashboard Title>",
      "tags": ["<category>"],
      "timezone": "browser",
      "schemaVersion": 39,
      "version": 1,
      "refresh": "30s",
      "editable": false,
      "graphTooltip": 2,
      "time": { "from": "now-24h", "to": "now" },
      "panels": [ ... ]
    }
```

> [!CAUTION]
> **No `"dashboard"` wrapper.** The JSON must have `uid`, `title`, `panels` at the top level. The `{"dashboard": {...}}` wrapper is only for the Grafana HTTP API — file provisioning via the sidecar will fail with *"Dashboard title cannot be empty"* if you use it.

For the full panel JSON schema, units, thresholds, overrides, and transformations, see the **`dashboarding`** skill's `references/json-schema.md`.

### Project-Specific Dashboard Conventions

- `"editable": false` — strict GitOps; never edit in the Grafana UI
- `"graphTooltip": 2` — shared crosshair + tooltip across all panels
- Use emoji in titles for visual appeal (🏠 ⚡ 🔥 🌡️ 💧 💨)
- Include `calcs` in legend: `["lastNotNull", "min", "max", "mean"]`
- Use `"spanNulls": true` to handle gaps in sensor data
- Use semantic threshold colors: blue=cold, green=comfort, orange=warm, red=hot

---

## Step 4: Deploy & Verify

```bash
# Apply
kubectl apply -f dashboards/grafana-dashboard-<name>.yaml

# Verify ConfigMap labeling
kubectl get configmaps -n monitoring -l grafana_dashboard=1

# Check sidecar pickup (container name is singular: grafana-sc-dashboard)
kubectl -n monitoring logs -l "app.kubernetes.io/name=grafana" \
  -c grafana-sc-dashboard --tail=20

# Check for Grafana errors
kubectl -n monitoring logs -l "app.kubernetes.io/name=grafana" \
  -c grafana --tail=30 | grep -i "error\|failed"
```

### Updating an Existing Dashboard

1. Edit the YAML file in `dashboards/`
2. Increment the `version` field in the dashboard JSON
3. `kubectl apply -f dashboards/grafana-dashboard-<name>.yaml`
4. Sidecar detects the change within ~60 seconds

### Removing a Dashboard

```bash
kubectl delete configmap -n monitoring grafana-dashboard-<name>
```

---

## InfluxDB Flux Queries

This section covers the **project-specific Flux conventions** for querying Home Assistant data in InfluxDB. These patterns are unique to this project and not covered by the Grafana skills.

### Data Schema

Home Assistant writes to InfluxDB with this structure:

- **Measurement name** = unit of measurement (e.g., `°C`, `%`, `W`, `kWh`)
- **Tags**: `entity_id` (e.g., `home_wohnzimmer_temperature`), `domain` (e.g., `sensor`)
- **Fields**: `value` (numeric), plus metadata strings (`friendly_name_str`, etc.)

### Critical Flux Query Rules

> [!WARNING]
> These rules come from hard-won debugging. Violating any of them will result in "no data" panels.

1. **`entity_id` has NO `sensor.` prefix** — HA stores the domain in a separate `domain` tag. Use `home_wohnzimmer_temperature`, NOT `sensor.home_wohnzimmer_temperature`.
2. **Always use bracket notation** for underscore-prefixed fields: `r["_measurement"]`, `r["_field"]` — dot notation (`r._measurement`) may silently fail.
3. **Always use Grafana time variables** — `v.timeRangeStart`, `v.timeRangeStop`, `v.windowPeriod`. Never hardcode `range(start: -24h)`.
4. **Always end with `|> yield(name: "mean")`** — required for Grafana to parse Flux results.
5. **Always filter `r["_field"] == "value"`** — InfluxDB stores metadata strings in other fields.
6. **Use regex for multi-entity panels** — `r.entity_id =~ /^home_/` instead of `or` chains (Flux `or` inside a single filter is unreliable).

### Flux Query Templates

**Time series panel (aggregated):**
```flux
from(bucket: "default")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "<UNIT>" and r["_field"] == "value")
  |> filter(fn: (r) => r.entity_id == "<ENTITY_ID>")
  |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false)
  |> yield(name: "mean")
```

**Stat panel (current value):**
```flux
from(bucket: "default")
  |> range(start: -15m)
  |> filter(fn: (r) => r["_measurement"] == "<UNIT>" and r["_field"] == "value")
  |> filter(fn: (r) => r.entity_id == "<ENTITY_ID>")
  |> last()
  |> yield(name: "mean")
```

**Multi-entity panel (regex):**
```flux
from(bucket: "default")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "<UNIT>" and r["_field"] == "value")
  |> filter(fn: (r) => r.entity_id =~ /^home_/)
  |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false)
  |> yield(name: "mean")
```

---

## InfluxDB Measurement Reference

| Measurement | Grafana Unit | Typical Entities |
|---|---|---|
| `°C` | `celsius` | Wandthermostat temperatures, outdoor temp, battery temp |
| `%` | `percent` | Humidity, battery levels, heating valve position, cloud cover |
| `W` | `watt` | Solar production, self-consumption, grid import/export |
| `kWh` | `kwatth` | Daily energy totals (production, consumption, import, export) |
| `m³` | `m3` | Gas volume counter |
| `m³/h` | `m3/h` | Current gas flow rate |
| `hPa` | `pressurehpa` | Barometric pressure |
| `km/h` | `velocitykmh` | Wind speed |
| `mm/h` | `lengthmm` | Precipitation rate |
| `EUR` | `currencyEUR` | Gas cost, electricity cost |