The incus scrape job (/1.0/metrics) exports both incus_* and node_* metrics,
duplicating the dedicated node-exporter containers. This caused 6 "nodes"
instead of 3 in dashboards, with the incus-sourced entries showing container
artifacts (epoch-0 boot time, 128 MiB RAM).
Prometheus config changes:
- Drop node_* metrics from incus job via metric_relabel_configs
- Relabel node-exporter instances to friendly names (oc-node-01/02/03)
using per-target nodename labels + relabel_configs
Dashboard changes (belt-and-suspenders):
- Add job="node-exporter" to all template variables and panel queries
in host-resources, network-deep-dive, storage-filesystem, capacity-planning
- Add Logs Drilldown link to logs-explorer dashboard
- Add Promtail journal relabels for level and service_name
Also: install Logs Drilldown plugin, update observability guide with
Logs Drilldown documentation and screenshots.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace incorrect Prometheus queries across 4 dashboard JSON files:
- incus_uptime_seconds → incus_cpu_effective_total for template vars and counts
- incus_memory_RSS_bytes → incus_memory_Active_bytes (RSS only reports for VMs)
- Uptime panel → Processes panel using incus_procs_total
- Remove wrong instance filter from network deep dive instance panels
- Fix Loki data directory permissions in deploy-observability
Add viewport screenshots of all 9 Grafana dashboards and embed them in
the observability guide with captions and updated descriptions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>