docs: add JSON logging audit to Loki improvements plan
Some checks failed
Run nix flake check / flake-check (push) Failing after 15m38s

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-02-13 22:44:05 +01:00
parent 1544415ef3
commit 2f0dad1acc

View File

@@ -125,7 +125,25 @@ This enables queries like:
- `{level=~"critical|error", tier="prod"}` - prod errors and criticals
- `{level="warning", role="dns"}` - warnings from DNS servers
### 6. Monitoring CNAME for Promtail Target
### 6. Enable JSON Logging on Services
**Problem:** Many services support structured JSON log output but may be using plain text by default. JSON logs are significantly easier to query in Loki - `| json` cleanly extracts all fields, whereas plain text requires fragile regex or pattern matching.
**Recommendation:** Audit all configured services and enable JSON logging where supported. Candidates to check include:
- Caddy (already JSON by default)
- Prometheus / Alertmanager / Loki / Tempo
- Grafana
- NSD / Unbound
- Home Assistant
- NATS
- Jellyfin
- OpenBao (Vault)
- Kanidm
- Garage
For each service, check whether it supports a JSON log format option and whether enabling it would break anything (e.g., log volume increase from verbose JSON, or dashboards that parse text format).
### 7. Monitoring CNAME for Promtail Target
**Problem:** Promtail hardcodes `monitoring01.home.2rjus.net:3100`. The VictoriaMetrics migration plan already addresses this by switching to a `monitoring` CNAME.
@@ -139,7 +157,8 @@ This enables queries like:
| 2 | **Limits config** | Low | Medium | Do with retention - minimal additional effort |
| 3 | **Promtail label fix** | Trivial | Low | Quick fix, do with other label changes |
| 4 | **Journal priority → level** | Low-medium | Medium | Reliable level filtering across the fleet |
| 5 | **Monitoring CNAME** | Low | Medium | Part of monitoring02 migration |
| 5 | **JSON logging audit** | Low-medium | Medium | Audit services, enable JSON where supported |
| 6 | **Monitoring CNAME** | Low | Medium | Part of monitoring02 migration |
## Implementation Steps