loki-monitoring02 #41

Merged

torjus merged 7 commits from loki-monitoring02 into master 2026-02-17 19:40:33 +00:00

Conversation 0 Commits 7 Files Changed 11 +160 -24

torjus commented 2026-02-17 19:40:27 +00:00

Owner

Copy Link

Summary

• Deploy a standalone Loki instance on monitoring02 with Caddy reverse proxy and basic auth, enabling dual-shipping of logs from all promtail clients
• Promtail now ships logs to both monitoring01 (direct) and loki.home.2rjus.net (basic auth via Vault secret), providing log redundancy on the new Loki instance
• Terraform policies updated to grant all hosts (including generated AppRoles) access to the shared loki-push secret
• Gitignore .mcp.json since it now contains Loki credentials; add .mcp.json.example with placeholders

Changes

• services/loki/ - New standalone Loki service module (same config as monitoring01)
• hosts/monitoring02/ - Import Loki service, add CNAME, Caddy reverse proxy with basic auth
• system/promtail.nix - Dual-ship logs to loki.home.2rjus.net with Vault-managed basic auth
• terraform/vault/ - Add loki-push policy to all AppRoles (both manual and generated)
• .mcp.json / .gitignore - Move MCP config to example file, gitignore real config

## Summary - Deploy a standalone Loki instance on monitoring02 with Caddy reverse proxy and basic auth, enabling dual-shipping of logs from all promtail clients - Promtail now ships logs to both monitoring01 (direct) and loki.home.2rjus.net (basic auth via Vault secret), providing log redundancy on the new Loki instance - Terraform policies updated to grant all hosts (including generated AppRoles) access to the shared loki-push secret - Gitignore `.mcp.json` since it now contains Loki credentials; add `.mcp.json.example` with placeholders ## Changes - `services/loki/` - New standalone Loki service module (same config as monitoring01) - `hosts/monitoring02/` - Import Loki service, add CNAME, Caddy reverse proxy with basic auth - `system/promtail.nix` - Dual-ship logs to loki.home.2rjus.net with Vault-managed basic auth - `terraform/vault/` - Add loki-push policy to all AppRoles (both manual and generated) - `.mcp.json` / `.gitignore` - Move MCP config to example file, gitignore real config

torjus added 7 commits 2026-02-17 19:40:27 +00:00

monitoring02: add Loki service ... 74e7c9faa4

Add standalone Loki service module (services/loki/) with same config as
monitoring01 and import it on monitoring02. Update Grafana Loki datasource
to localhost. Defer Tempo and Pyroscope migration (not actively used).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

monitoring02: add loki CNAME and Caddy reverse proxy ... 2903873d52

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

loki: add basic auth for log push and dual-ship promtail ... c13921d302

- Loki bound to localhost, Caddy reverse proxy with basic_auth
- Vault secret (shared/loki/push-auth) for password, bcrypt hash
  generated at boot for Caddy environment
- Promtail dual-ships to monitoring01 (direct) and loki.home.2rjus.net
  (with basic auth), conditional on vault.enable
- Terraform: new shared loki-push policy added to all AppRoles

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

terraform: add ns1 and ns2 to AppRole policies ... 58f901ad3e

They were missing from the host_policies map, so they didn't get
shared policies like loki-push.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

terraform: fix loki-push policy for generated hosts ... 43c81f6688

Revert ns1/ns2 from approle.tf (they're in hosts-generated.tf) and add
loki-push policy to generated AppRoles instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

promtail: fix vault secret ownership for loki auth ... 87d8571d62

The secret file needs to be owned by promtail since Promtail runs
as a dedicated user and can't read root-owned files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mcp: move config to .mcp.json.example, gitignore real config ... 35924c7b01

The real .mcp.json now contains Loki credentials for basic auth,
so it should not be committed. The example file has placeholders.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

torjus merged commit 7f69c0738a into master 2026-02-17 19:40:33 +00:00

torjus referenced this issue from a commit 2026-02-17 19:40:35 +00:00

Merge pull request 'loki-monitoring02' (#41) from loki-monitoring02 into master

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

torjus

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: torjus/nixos-servers#41