decommission-ca-host #32

Merged
torjus merged 3 commits from decommission-ca-host into master 2026-02-07 17:50:45 +00:00
13 changed files with 10 additions and 325 deletions
Showing only changes of commit bdc6057689 - Show all commits

View File

@@ -92,7 +92,7 @@ Secrets are managed by OpenBao (Vault) using AppRole authentication. Most hosts
`vault.secrets` option defined in `system/vault-secrets.nix` to fetch secrets at boot. `vault.secrets` option defined in `system/vault-secrets.nix` to fetch secrets at boot.
Terraform manages the secrets and AppRole policies in `terraform/vault/`. Terraform manages the secrets and AppRole policies in `terraform/vault/`.
Legacy sops-nix is still present but only actively used by the `ca` host. Do not edit any Legacy sops-nix is still present but no longer actively used. Do not edit any
`.sops.yaml` or any file within `secrets/`. Ask the user to modify if necessary. `.sops.yaml` or any file within `secrets/`. Ask the user to modify if necessary.
### Git Workflow ### Git Workflow
@@ -210,7 +210,6 @@ The **lab-monitoring** MCP server can query Prometheus metrics via PromQL. The `
- `home-assistant` - Home automation metrics - `home-assistant` - Home automation metrics
- `jellyfin` - Media server metrics - `jellyfin` - Media server metrics
- `loki` / `prometheus` / `grafana` - Monitoring stack self-metrics - `loki` / `prometheus` / `grafana` - Monitoring stack self-metrics
- `step-ca` - Internal CA metrics
- `pve-exporter` - Proxmox hypervisor metrics - `pve-exporter` - Proxmox hypervisor metrics
- `smartctl` - Disk SMART health (gunter) - `smartctl` - Disk SMART health (gunter)
- `wireguard` - VPN metrics (http-proxy) - `wireguard` - VPN metrics (http-proxy)
@@ -316,14 +315,14 @@ The `current_rev` label contains the git commit hash of the deployed flake confi
- `ns/` - DNS services (authoritative, resolver, zone generation) - `ns/` - DNS services (authoritative, resolver, zone generation)
- `vault/` - OpenBao (Vault) secrets server - `vault/` - OpenBao (Vault) secrets server
- `actions-runner/` - GitHub Actions runner - `actions-runner/` - GitHub Actions runner
- `http-proxy/`, `ca/`, `postgres/`, `nats/`, `jellyfin/`, etc. - `http-proxy/`, `postgres/`, `nats/`, `jellyfin/`, etc.
- `/secrets/` - SOPS-encrypted secrets with age encryption (legacy, only used by ca) - `/secrets/` - SOPS-encrypted secrets with age encryption (legacy, no longer used)
- `/common/` - Shared configurations (e.g., VM guest agent) - `/common/` - Shared configurations (e.g., VM guest agent)
- `/docs/` - Documentation and plans - `/docs/` - Documentation and plans
- `plans/` - Future plans and proposals - `plans/` - Future plans and proposals
- `plans/completed/` - Completed plans (moved here when done) - `plans/completed/` - Completed plans (moved here when done)
- `/playbooks/` - Ansible playbooks for fleet management - `/playbooks/` - Ansible playbooks for fleet management
- `/.sops.yaml` - SOPS configuration with age keys (legacy, only used by ca) - `/.sops.yaml` - SOPS configuration with age keys (legacy, no longer used)
### Configuration Inheritance ### Configuration Inheritance
@@ -340,7 +339,7 @@ All hosts automatically get:
- Nix binary cache (nix-cache.home.2rjus.net) - Nix binary cache (nix-cache.home.2rjus.net)
- SSH with root login enabled - SSH with root login enabled
- OpenBao (Vault) secrets management via AppRole - OpenBao (Vault) secrets management via AppRole
- Internal ACME CA integration (ca.home.2rjus.net) - Internal ACME CA integration (OpenBao PKI at vault.home.2rjus.net)
- Daily auto-upgrades with auto-reboot - Daily auto-upgrades with auto-reboot
- Prometheus node-exporter + Promtail (logs to monitoring01) - Prometheus node-exporter + Promtail (logs to monitoring01)
- Monitoring scrape target auto-registration via `homelab.monitoring` options - Monitoring scrape target auto-registration via `homelab.monitoring` options
@@ -351,8 +350,7 @@ All hosts automatically get:
Production servers: Production servers:
- `ns1`, `ns2` - Primary/secondary DNS servers (10.69.13.5/6) - `ns1`, `ns2` - Primary/secondary DNS servers (10.69.13.5/6)
- `ca` - Internal Certificate Authority - `vault01` - OpenBao (Vault) secrets server + PKI CA
- `vault01` - OpenBao (Vault) secrets server
- `ha1` - Home Assistant + Zigbee2MQTT + Mosquitto - `ha1` - Home Assistant + Zigbee2MQTT + Mosquitto
- `http-proxy` - Reverse proxy - `http-proxy` - Reverse proxy
- `monitoring01` - Full observability stack (Prometheus, Grafana, Loki, Tempo, Pyroscope) - `monitoring01` - Full observability stack (Prometheus, Grafana, Loki, Tempo, Pyroscope)
@@ -371,7 +369,7 @@ Template hosts:
- `nixpkgs` - NixOS 25.11 stable (primary) - `nixpkgs` - NixOS 25.11 stable (primary)
- `nixpkgs-unstable` - Unstable channel (available via overlay as `pkgs.unstable.<package>`) - `nixpkgs-unstable` - Unstable channel (available via overlay as `pkgs.unstable.<package>`)
- `sops-nix` - Secrets management (legacy, only used by ca) - `sops-nix` - Secrets management (legacy, no longer actively used)
- `nixos-exporter` - NixOS module for exposing flake revision metrics (used to verify deployments) - `nixos-exporter` - NixOS module for exposing flake revision metrics (used to verify deployments)
- `homelab-deploy` - NATS-based remote deployment tool for test-tier hosts - `homelab-deploy` - NATS-based remote deployment tool for test-tier hosts
- Custom packages from git.t-juice.club: - Custom packages from git.t-juice.club:
@@ -399,10 +397,9 @@ Most hosts use OpenBao (Vault) for secrets:
- Fallback to cached secrets in `/var/lib/vault/cache/` when Vault is unreachable - Fallback to cached secrets in `/var/lib/vault/cache/` when Vault is unreachable
- Provision AppRole credentials: `nix develop -c ansible-playbook playbooks/provision-approle.yml -e hostname=<host>` - Provision AppRole credentials: `nix develop -c ansible-playbook playbooks/provision-approle.yml -e hostname=<host>`
Legacy SOPS (only used by `ca` host): Legacy SOPS (no longer actively used):
- SOPS with age encryption, keys in `.sops.yaml` - SOPS with age encryption, keys in `.sops.yaml`
- Shared secrets: `/secrets/secrets.yaml` - Files in `/secrets/` are legacy and can be removed
- Per-host secrets: `/secrets/<hostname>/`
### Auto-Upgrade System ### Auto-Upgrade System
@@ -558,7 +555,7 @@ Prometheus scrape targets are automatically generated from host configurations,
- **External targets**: Non-flake hosts defined in `/services/monitoring/external-targets.nix` - **External targets**: Non-flake hosts defined in `/services/monitoring/external-targets.nix`
- **Library**: `lib/monitoring.nix` provides `generateNodeExporterTargets` and `generateScrapeConfigs` - **Library**: `lib/monitoring.nix` provides `generateNodeExporterTargets` and `generateScrapeConfigs`
Service modules declare their scrape targets directly via `homelab.monitoring.scrapeTargets` (e.g., `services/ca/default.nix` declares step-ca on port 9000). The Prometheus config on monitoring01 auto-generates scrape configs from all hosts. See "Homelab Module Options" section for available options. Service modules declare their scrape targets directly via `homelab.monitoring.scrapeTargets`. The Prometheus config on monitoring01 auto-generates scrape configs from all hosts. See "Homelab Module Options" section for available options.
To add monitoring targets for non-NixOS hosts, edit `/services/monitoring/external-targets.nix`. To add monitoring targets for non-NixOS hosts, edit `/services/monitoring/external-targets.nix`.

22
flake.lock generated
View File

@@ -42,27 +42,6 @@
"url": "https://git.t-juice.club/torjus/homelab-deploy" "url": "https://git.t-juice.club/torjus/homelab-deploy"
} }
}, },
"labmon": {
"inputs": {
"nixpkgs": [
"nixpkgs-unstable"
]
},
"locked": {
"lastModified": 1748983975,
"narHash": "sha256-DA5mOqxwLMj/XLb4hvBU1WtE6cuVej7PjUr8N0EZsCE=",
"ref": "master",
"rev": "040a73e891a70ff06ec7ab31d7167914129dbf7d",
"revCount": 17,
"type": "git",
"url": "https://git.t-juice.club/torjus/labmon"
},
"original": {
"ref": "master",
"type": "git",
"url": "https://git.t-juice.club/torjus/labmon"
}
},
"nixos-exporter": { "nixos-exporter": {
"inputs": { "inputs": {
"nixpkgs": [ "nixpkgs": [
@@ -119,7 +98,6 @@
"inputs": { "inputs": {
"alerttonotify": "alerttonotify", "alerttonotify": "alerttonotify",
"homelab-deploy": "homelab-deploy", "homelab-deploy": "homelab-deploy",
"labmon": "labmon",
"nixos-exporter": "nixos-exporter", "nixos-exporter": "nixos-exporter",
"nixpkgs": "nixpkgs", "nixpkgs": "nixpkgs",
"nixpkgs-unstable": "nixpkgs-unstable", "nixpkgs-unstable": "nixpkgs-unstable",

View File

@@ -13,10 +13,6 @@
url = "git+https://git.t-juice.club/torjus/alerttonotify?ref=master"; url = "git+https://git.t-juice.club/torjus/alerttonotify?ref=master";
inputs.nixpkgs.follows = "nixpkgs-unstable"; inputs.nixpkgs.follows = "nixpkgs-unstable";
}; };
labmon = {
url = "git+https://git.t-juice.club/torjus/labmon?ref=master";
inputs.nixpkgs.follows = "nixpkgs-unstable";
};
nixos-exporter = { nixos-exporter = {
url = "git+https://git.t-juice.club/torjus/nixos-exporter"; url = "git+https://git.t-juice.club/torjus/nixos-exporter";
inputs.nixpkgs.follows = "nixpkgs-unstable"; inputs.nixpkgs.follows = "nixpkgs-unstable";
@@ -34,7 +30,6 @@
nixpkgs-unstable, nixpkgs-unstable,
sops-nix, sops-nix,
alerttonotify, alerttonotify,
labmon,
nixos-exporter, nixos-exporter,
homelab-deploy, homelab-deploy,
... ...
@@ -50,7 +45,6 @@
commonOverlays = [ commonOverlays = [
overlay-unstable overlay-unstable
alerttonotify.overlays.default alerttonotify.overlays.default
labmon.overlays.default
]; ];
# Common modules applied to all hosts # Common modules applied to all hosts
commonModules = [ commonModules = [
@@ -131,15 +125,6 @@
./hosts/http-proxy ./hosts/http-proxy
]; ];
}; };
ca = nixpkgs.lib.nixosSystem {
inherit system;
specialArgs = {
inherit inputs self sops-nix;
};
modules = commonModules ++ [
./hosts/ca
];
};
monitoring01 = nixpkgs.lib.nixosSystem { monitoring01 = nixpkgs.lib.nixosSystem {
inherit system; inherit system;
specialArgs = { specialArgs = {
@@ -147,7 +132,6 @@
}; };
modules = commonModules ++ [ modules = commonModules ++ [
./hosts/monitoring01 ./hosts/monitoring01
labmon.nixosModules.labmon
]; ];
}; };
jelly01 = nixpkgs.lib.nixosSystem { jelly01 = nixpkgs.lib.nixosSystem {

View File

@@ -1,63 +0,0 @@
{
pkgs,
...
}:
{
imports = [
../template/hardware-configuration.nix
../../system
../../common/vm
];
nixpkgs.config.allowUnfree = true;
# Use the systemd-boot EFI boot loader.
boot.loader.grub = {
enable = true;
device = "/dev/sda";
configurationLimit = 3;
};
networking.hostName = "ca";
networking.domain = "home.2rjus.net";
networking.useNetworkd = true;
networking.useDHCP = false;
services.resolved.enable = true;
networking.nameservers = [
"10.69.13.5"
"10.69.13.6"
];
systemd.network.enable = true;
systemd.network.networks."ens18" = {
matchConfig.Name = "ens18";
address = [
"10.69.13.12/24"
];
routes = [
{ Gateway = "10.69.13.1"; }
];
linkConfig.RequiredForOnline = "routable";
};
time.timeZone = "Europe/Oslo";
nix.settings.experimental-features = [
"nix-command"
"flakes"
];
nix.settings.tarball-ttl = 0;
environment.systemPackages = with pkgs; [
vim
wget
git
];
# Open ports in the firewall.
# networking.firewall.allowedTCPPorts = [ ... ];
# networking.firewall.allowedUDPPorts = [ ... ];
# Or disable the firewall altogether.
networking.firewall.enable = false;
system.stateVersion = "23.11"; # Did you read the comment?
}

View File

@@ -1,7 +0,0 @@
{ ... }:
{
imports = [
./configuration.nix
../../services/ca
];
}

View File

@@ -5,7 +5,6 @@ set -euo pipefail
HOSTS=( HOSTS=(
"ns1" "ns1"
"ns2" "ns2"
"ca"
"ha1" "ha1"
"http-proxy" "http-proxy"
"jelly01" "jelly01"

View File

@@ -1,169 +0,0 @@
{ pkgs, unstable, ... }:
{
homelab.monitoring.scrapeTargets = [{
job_name = "step-ca";
port = 9000;
}];
sops.secrets."ca_root_pw" = {
sopsFile = ../../secrets/ca/secrets.yaml;
owner = "step-ca";
path = "/var/lib/step-ca/secrets/ca_root_pw";
};
sops.secrets."intermediate_ca_key" = {
sopsFile = ../../secrets/ca/keys/intermediate_ca_key;
format = "binary";
owner = "step-ca";
path = "/var/lib/step-ca/secrets/intermediate_ca_key";
};
sops.secrets."root_ca_key" = {
sopsFile = ../../secrets/ca/keys/root_ca_key;
format = "binary";
owner = "step-ca";
path = "/var/lib/step-ca/secrets/root_ca_key";
};
sops.secrets."ssh_host_ca_key" = {
sopsFile = ../../secrets/ca/keys/ssh_host_ca_key;
format = "binary";
owner = "step-ca";
path = "/var/lib/step-ca/secrets/ssh_host_ca_key";
};
sops.secrets."ssh_user_ca_key" = {
sopsFile = ../../secrets/ca/keys/ssh_user_ca_key;
format = "binary";
owner = "step-ca";
path = "/var/lib/step-ca/secrets/ssh_user_ca_key";
};
services.step-ca = {
enable = true;
package = pkgs.step-ca;
intermediatePasswordFile = "/var/lib/step-ca/secrets/ca_root_pw";
address = "0.0.0.0";
port = 443;
settings = {
metricsAddress = ":9000";
authority = {
provisioners = [
{
claims = {
enableSSHCA = true;
maxTLSCertDuration = "3600h";
defaultTLSCertDuration = "48h";
};
encryptedKey = "eyJhbGciOiJQQkVTMi1IUzI1NitBMTI4S1ciLCJjdHkiOiJqd2sranNvbiIsImVuYyI6IkEyNTZHQ00iLCJwMmMiOjYwMDAwMCwicDJzIjoiY1lWOFJPb3lteXFLMWpzcS1WM1ZXQSJ9.WS8tPK-Q4gtnSsw7MhpTzYT_oi-SQx-CsRLh7KwdZnpACtd4YbcOYg.zeyDkmKRx8BIp-eB.OQ8c-KDW07gqJFtEMqHacRBkttrbJRRz0sYR47vQWDCoWhodaXsxM_Bj2pGvUrR26ij1t7irDeypnJoh6WXvUg3n_JaIUL4HgTwKSBrXZKTscXmY7YVmRMionhAb6oS9Jgus9K4QcFDHacC9_WgtGI7dnu3m0G7c-9Ur9dcDfROfyrnAByJp1rSZMzvriQr4t9bNYjDa8E8yu9zq6aAQqF0Xg_AxwiqYqesT-sdcfrxKS61appApRgPlAhW-uuzyY0wlWtsiyLaGlWM7WMfKdHsq-VqcVrI7Gi2i77vi7OqPEberqSt8D04tIri9S_sArKqWEDnBJsL07CC41IY.CqtYfbSa_wlmIsKgNj5u7g";
key = {
alg = "ES256";
crv = "P-256";
kid = "CIjtIe7FNhsNQe1qKGD9Rpj-lrf2ExyTYCXAOd3YDjE";
kty = "EC";
use = "sig";
x = "XRMX-BeobZ-R5-xb-E9YlaRjJUfd7JQxpscaF1NMgFo";
y = "bF9xLp5-jywRD-MugMaOGbpbniPituWSLMlXRJnUUl0";
};
name = "ca@home.2rjus.net";
type = "JWK";
}
{
name = "acme";
type = "ACME";
claims = {
maxTLSCertDuration = "3600h";
defaultTLSCertDuration = "1800h";
};
}
{
claims = {
enableSSHCA = true;
};
name = "sshpop";
type = "SSHPOP";
}
];
};
crt = "/var/lib/step-ca/certs/intermediate_ca.crt";
db = {
badgerFileLoadingMode = "";
dataSource = "/var/lib/step-ca/db";
type = "badgerv2";
};
dnsNames = [
"ca.home.2rjus.net"
"10.69.13.12"
];
federatedRoots = null;
insecureAddress = "";
key = "/var/lib/step-ca/secrets/intermediate_ca_key";
logger = {
format = "text";
};
root = "/var/lib/step-ca/certs/root_ca.crt";
ssh = {
hostKey = "/var/lib/step-ca/secrets/ssh_host_ca_key";
userKey = "/var/lib/step-ca/secrets/ssh_user_ca_key";
};
templates = {
ssh = {
host = [
{
comment = "#";
name = "sshd_config.tpl";
path = "/etc/ssh/sshd_config";
requires = [
"Certificate"
"Key"
];
template = ./templates/ssh/sshd_config.tpl;
type = "snippet";
}
{
comment = "#";
name = "ca.tpl";
path = "/etc/ssh/ca.pub";
template = ./templates/ssh/ca.tpl;
type = "snippet";
}
];
user = [
{
comment = "#";
name = "config.tpl";
path = "~/.ssh/config";
template = ./templates/ssh/config.tpl;
type = "snippet";
}
{
comment = "#";
name = "step_includes.tpl";
path = "\${STEPPATH}/ssh/includes";
template = ./templates/ssh/step_includes.tpl;
type = "prepend-line";
}
{
comment = "#";
name = "step_config.tpl";
path = "ssh/config";
template = ./templates/ssh/step_config.tpl;
type = "file";
}
{
comment = "#";
name = "known_hosts.tpl";
path = "ssh/known_hosts";
template = ./templates/ssh/known_hosts.tpl;
type = "file";
}
];
};
};
tls = {
cipherSuites = [
"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256"
"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256"
];
maxVersion = 1.3;
minVersion = 1.2;
renegotiation = false;
};
};
};
}

Binary file not shown.

View File

@@ -1,14 +0,0 @@
Host *
{{- if or .User.GOOS "none" | eq "windows" }}
{{- if .User.StepBasePath }}
Include "{{ .User.StepBasePath | replace "\\" "/" | trimPrefix "C:" }}/ssh/includes"
{{- else }}
Include "{{ .User.StepPath | replace "\\" "/" | trimPrefix "C:" }}/ssh/includes"
{{- end }}
{{- else }}
{{- if .User.StepBasePath }}
Include "{{.User.StepBasePath}}/ssh/includes"
{{- else }}
Include "{{.User.StepPath}}/ssh/includes"
{{- end }}
{{- end }}

View File

@@ -1,4 +0,0 @@
@cert-authority * {{.Step.SSH.HostKey.Type}} {{.Step.SSH.HostKey.Marshal | toString | b64enc}}
{{- range .Step.SSH.HostFederatedKeys}}
@cert-authority * {{.Type}} {{.Marshal | toString | b64enc}}
{{- end }}

View File

@@ -1,4 +0,0 @@
Match all
TrustedUserCAKeys /etc/ssh/ca.pub
HostCertificate /etc/ssh/{{.User.Certificate}}
HostKey /etc/ssh/{{.User.Key}}

View File

@@ -1,11 +0,0 @@
Match exec "step ssh check-host{{- if .User.Context }} --context {{ .User.Context }}{{- end }} %h"
{{- if .User.User }}
User {{.User.User}}
{{- end }}
{{- if or .User.GOOS "none" | eq "windows" }}
UserKnownHostsFile "{{.User.StepPath}}\ssh\known_hosts"
ProxyCommand C:\Windows\System32\cmd.exe /c step ssh proxycommand{{- if .User.Context }} --context {{ .User.Context }}{{- end }}{{- if .User.Provisioner }} --provisioner {{ .User.Provisioner }}{{- end }} %r %h %p
{{- else }}
UserKnownHostsFile "{{.User.StepPath}}/ssh/known_hosts"
ProxyCommand step ssh proxycommand{{- if .User.Context }} --context {{ .User.Context }}{{- end }}{{- if .User.Provisioner }} --provisioner {{ .User.Provisioner }}{{- end }} %r %h %p
{{- end }}

View File

@@ -1 +0,0 @@
{{- if or .User.GOOS "none" | eq "windows" }}Include "{{ .User.StepPath | replace "\\" "/" | trimPrefix "C:" }}/ssh/config"{{- else }}Include "{{.User.StepPath}}/ssh/config"{{- end }}