Torjus Håkestad
4f593126c0
Remove monitoring01 host configuration and unused service modules (prometheus, grafana, loki, tempo, pyroscope). Migrate blackbox, exportarr, and pve exporters to monitoring02 with scrape configs moved to VictoriaMetrics. Update alert rules, terraform vault policies/secrets, http-proxy entries, and documentation to reflect the monitoring02 migration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
125 lines
4.6 KiB
Markdown
125 lines
4.6 KiB
Markdown
# nixos-servers
|
|
|
|
NixOS Flake-based configuration repository for a homelab infrastructure. All hosts run NixOS 25.11 and are managed declaratively through this single repository.
|
|
|
|
## Hosts
|
|
|
|
| Host | Role |
|
|
|------|------|
|
|
| `ns1`, `ns2` | Primary/secondary authoritative DNS |
|
|
| `ca` | Internal Certificate Authority |
|
|
| `ha1` | Home Assistant + Zigbee2MQTT + Mosquitto |
|
|
| `http-proxy` | Reverse proxy |
|
|
| `monitoring02` | VictoriaMetrics, Grafana, Loki, Alertmanager |
|
|
| `jelly01` | Jellyfin media server |
|
|
| `nix-cache02` | Nix binary cache + NATS-based build service |
|
|
| `nats1` | NATS messaging |
|
|
| `vault01` | OpenBao (Vault) secrets management |
|
|
| `template1`, `template2` | VM templates for cloning new hosts |
|
|
|
|
## Directory Structure
|
|
|
|
```
|
|
flake.nix # Flake entry point, defines all host configurations
|
|
hosts/<hostname>/ # Per-host configuration
|
|
system/ # Shared modules applied to ALL hosts
|
|
services/ # Reusable service modules, selectively imported per host
|
|
modules/ # Custom NixOS module definitions
|
|
lib/ # Nix library functions (DNS zone generation, etc.)
|
|
secrets/ # SOPS-encrypted secrets (legacy, only used by ca)
|
|
common/ # Shared configurations (e.g., VM guest agent)
|
|
terraform/ # OpenTofu configs for Proxmox VM provisioning
|
|
terraform/vault/ # OpenTofu configs for OpenBao (secrets, PKI, AppRoles)
|
|
playbooks/ # Ansible playbooks for template building and fleet ops
|
|
scripts/ # Helper scripts (create-host, vault-fetch)
|
|
```
|
|
|
|
## Key Features
|
|
|
|
**Automatic DNS zone generation** - A records are derived from each host's static IP configuration. CNAME aliases are defined via `homelab.dns.cnames`. No manual zone file editing required.
|
|
|
|
**OpenBao (Vault) secrets** - Hosts authenticate via AppRole and fetch secrets at boot. Secrets and policies are managed as code in `terraform/vault/`. Legacy SOPS remains only for the `ca` host.
|
|
|
|
**Daily auto-upgrades** - All hosts pull from the master branch and automatically rebuild and reboot on a randomized schedule.
|
|
|
|
**Shared base configuration** - Every host automatically gets SSH, monitoring (node-exporter + Promtail), internal ACME certificates, and Nix binary cache access via the `system/` modules.
|
|
|
|
**Proxmox VM provisioning** - Build VM templates with Ansible and deploy VMs with OpenTofu from `terraform/`.
|
|
|
|
**OpenBao (Vault) secrets** - Centralized secrets management with AppRole authentication, PKI infrastructure, and automated bootstrap. Managed as code in `terraform/vault/`.
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
# Enter dev shell (provides ansible, opentofu, openbao, create-host)
|
|
nix develop
|
|
|
|
# Build a host configuration locally
|
|
nix build .#nixosConfigurations.<hostname>.config.system.build.toplevel
|
|
|
|
# List all configurations
|
|
nix flake show
|
|
```
|
|
|
|
Deployments are done by merging to master and triggering the auto-upgrade on the target host.
|
|
|
|
## Provisioning New Hosts
|
|
|
|
The repository includes an automated pipeline for creating and deploying new hosts on Proxmox.
|
|
|
|
### 1. Generate host configuration
|
|
|
|
The `create-host` tool (available in the dev shell) generates all required files for a new host:
|
|
|
|
```bash
|
|
create-host \
|
|
--hostname myhost \
|
|
--ip 10.69.13.50/24 \
|
|
--cpu 4 \
|
|
--memory 4096 \
|
|
--disk 50G
|
|
```
|
|
|
|
This creates:
|
|
- `hosts/<hostname>/` - NixOS configuration (networking, imports, hardware)
|
|
- Entry in `flake.nix`
|
|
- VM definition in `terraform/vms.tf`
|
|
- Vault AppRole policy and wrapped bootstrap token
|
|
|
|
Omit `--ip` for DHCP. Use `--dry-run` to preview changes. Use `--force` to regenerate an existing host's config.
|
|
|
|
### 2. Build and deploy the VM template
|
|
|
|
The Proxmox VM template is built from `hosts/template2` and deployed with Ansible:
|
|
|
|
```bash
|
|
nix develop -c ansible-playbook -i playbooks/inventory.ini playbooks/build-and-deploy-template.yml
|
|
```
|
|
|
|
This only needs to be re-run when the base template changes.
|
|
|
|
### 3. Deploy the VM
|
|
|
|
```bash
|
|
cd terraform && tofu apply
|
|
```
|
|
|
|
### 4. Automatic bootstrap
|
|
|
|
On first boot, the VM automatically:
|
|
1. Receives its hostname and Vault credentials via cloud-init
|
|
2. Unwraps the Vault token and stores AppRole credentials
|
|
3. Runs `nixos-rebuild boot` against the flake on the master branch
|
|
4. Reboots into the host-specific configuration
|
|
5. Services fetch their secrets from Vault at startup
|
|
|
|
No manual intervention is required after `tofu apply`.
|
|
|
|
## Network
|
|
|
|
- Domain: `home.2rjus.net`
|
|
- Infrastructure subnet: `10.69.13.0/24`
|
|
- DNS: ns1/ns2 authoritative with primary-secondary AXFR
|
|
- Internal CA for TLS certificates (migrating from step-ca to OpenBao PKI)
|
|
- Centralized monitoring at monitoring02
|