From 5e92eb32203896ce6f14b53522d4bb7e3446f126 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Torjus=20H=C3=A5kestad?= Date: Tue, 24 Feb 2026 00:41:50 +0100 Subject: [PATCH] docs: add plan for NixOS OpenStack image Co-Authored-By: Claude Opus 4.6 --- docs/plans/openstack-nixos-image.md | 104 ++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 docs/plans/openstack-nixos-image.md diff --git a/docs/plans/openstack-nixos-image.md b/docs/plans/openstack-nixos-image.md new file mode 100644 index 0000000..7a39e27 --- /dev/null +++ b/docs/plans/openstack-nixos-image.md @@ -0,0 +1,104 @@ +# NixOS OpenStack Image + +## Overview + +Build and upload a NixOS base image to the OpenStack cluster at work, enabling NixOS-based VPS instances to replace the current Debian+Podman setup. This image will serve as the foundation for multiple external services: + +- **Forgejo** (replacing Gitea on docker2) +- **WireGuard gateway** (replacing docker2's tunnel role, feeding into the remote-access plan) +- Any future externally-hosted services + +## Current State + +- VPS hosting runs on an OpenStack cluster with a personal quota +- Current VPS (`docker2.t-juice.club`) runs Debian with Podman containers +- Homelab already has a working Proxmox image pipeline: `template2` builds via `nixos-rebuild build-image --image-variant proxmox`, deployed via Ansible +- nixpkgs has a built-in `openstack` image variant in the same `image.modules` system used for Proxmox + +## Decisions + +- **No cloud-init dependency** - SSH key baked into the image, no need for metadata service +- **No bootstrap script** - VPS deployments are infrequent; manual `nixos-rebuild` after first boot is fine +- **No Vault access** - secrets handled manually until WireGuard access is set up (see remote-access plan) +- **Separate from homelab services** - no logging/metrics integration initially; revisit after remote-access WireGuard is in place +- **Repo placement TBD** - keep in this flake for now for convenience, but external hosts may move to a separate flake later since they can't use most shared `system/` modules (no Vault, no internal DNS, no Promtail) +- **OpenStack CLI in devshell** - add `openstackclient` package; credentials (`clouds.yaml`) stay outside the repo +- **Parallel deployment** - new Forgejo instance runs alongside docker2 initially, then CNAME moves over + +## Approach + +Follow the same pattern as the Proxmox template (`hosts/template2`), but targeting OpenStack's qcow2 format. + +### What nixpkgs provides + +The `image.modules.openstack` module produces a qcow2 image with: +- `openstack-config.nix`: EC2 metadata fetcher, SSH enabled, GRUB bootloader, serial console, auto-growing root partition +- `qemu-guest.nix` profile (virtio drivers) +- ext4 root filesystem with `autoResize` + +### What we need to customize + +The stock OpenStack image pulls SSH keys and hostname from EC2-style metadata. Since we're baking the SSH key into the image, we need a simpler configuration: + +- SSH authorized keys baked into the image +- Base packages (age, vim, wget, git) +- Nix substituters (`cache.nixos.org` only - internal cache not reachable) +- systemd-networkd with DHCP +- GRUB bootloader +- Firewall enabled (public-facing host) + +### Differences from template2 + +| Aspect | template2 (Proxmox) | openstack-template (OpenStack) | +|--------|---------------------|-------------------------------| +| Image format | VMA (`.vma.zst`) | qcow2 (`.qcow2`) | +| Image variant | `proxmox` | `openstack` | +| Cloud-init | ConfigDrive + NoCloud | Not used (SSH key baked in) | +| Nix cache | Internal + nixos.org | `cache.nixos.org` only | +| Vault | AppRole via wrapped token | None | +| Bootstrap | Automatic nixos-rebuild on first boot | Manual | +| Network | Internal DHCP | OpenStack DHCP | +| DNS | Internal ns1/ns2 | Public DNS | +| Firewall | Disabled (trusted network) | Enabled | +| System modules | Full `../../system` import | Minimal (sshd, packages only) | + +## Implementation Steps + +### Phase 1: Build the image + +1. Create `hosts/openstack-template/` with minimal configuration + - `default.nix` - imports (only sshd and packages from `system/`, not the full set) + - `configuration.nix` - base config: SSH key, DHCP, GRUB, base packages, firewall on + - `hardware-configuration.nix` - qemu-guest profile with virtio drivers + - Exclude from DNS and monitoring (`homelab.dns.enable = false`, `homelab.monitoring.enable = false`) + - May need to override parts of `image.modules.openstack` to disable the EC2 metadata fetcher if it causes boot delays +2. Build with `nixos-rebuild build-image --image-variant openstack --flake .#openstack-template` +3. Verify the qcow2 image is produced in `result/` + +### Phase 2: Upload and test + +1. Add `openstackclient` to the devshell +2. Upload image: `openstack image create --disk-format qcow2 --file result/.qcow2 nixos-template` +3. Boot a test instance from the image +4. Verify: SSH access works, DHCP networking, Nix builds work +5. Test manual `nixos-rebuild switch --flake` against the instance + +### Phase 3: Automation (optional, later) + +Consider an Ansible playbook similar to `build-and-deploy-template.yml` for image builds + uploads. Low priority since this will be done rarely. + +## Open Questions + +- [ ] Should external VPS hosts eventually move to a separate flake? (Depends on how different they end up being from homelab hosts) +- [ ] Will the stock `openstack-config.nix` metadata fetcher cause boot delays/errors if the metadata service isn't reachable? May need to disable it. +- [ ] **Flavor selection** - investigate what flavors are available in the quota. The standard small flavors likely have insufficient root disk for a NixOS host (Nix store grows fast). Options: + - Use a larger flavor with adequate root disk + - Create a custom flavor (if permissions allow) + - Cinder block storage is an option in theory, but was very slow last time it was tested - avoid if possible +- [ ] Consolidation opportunity - currently running multiple smaller VMs on OpenStack. Could a single larger NixOS VM replace several of them? + +## Notes + +- `nixos-rebuild build-image --image-variant openstack` uses the same `image.modules` system as Proxmox +- nixpkgs also has an `openstack-zfs` variant if ZFS root is ever wanted +- The stock OpenStack module imports `ec2-data.nix` and `amazon-init.nix` - these may need to be disabled or overridden if they cause issues without a metadata service