Merge pull request 'dns-automation' (#15) from dns-automation into master

Reviewed-on: #15

CLAUDE.md

@@ -54,6 +54,12 @@ nix develop
 
 Secrets are handled by sops. Do not edit any `.sops.yaml` or any file within `secrets/`. Ask the user to modify if necessary.
 
+### Git Workflow
+
+**Important:** Never commit directly to `master` unless the user explicitly asks for it. Always create a feature branch for changes.
+
+When starting a new plan or task, the first step should typically be to create and checkout a new branch with an appropriate name (e.g., `git checkout -b dns-automation` or `git checkout -b fix-nginx-config`).
+
 ### Git Commit Messages
 
 Commit messages should follow the format: `topic: short description`
@@ -101,17 +107,21 @@ This ensures documentation matches the exact nixpkgs version (currently NixOS 25
 
 ### Directory Structure
 
-- `/flake.nix` - Central flake defining all 16 NixOS configurations
+- `/flake.nix` - Central flake defining all NixOS configurations
 - `/hosts/<hostname>/` - Per-host configurations
   - `default.nix` - Entry point, imports configuration.nix and services
   - `configuration.nix` - Host-specific settings (networking, hardware, users)
 - `/system/` - Shared system-level configurations applied to ALL hosts
   - Core modules: nix.nix, sshd.nix, sops.nix, acme.nix, autoupgrade.nix
   - Monitoring: node-exporter and promtail on every host
+- `/modules/` - Custom NixOS modules
+  - `homelab/` - Homelab-specific options (DNS automation, etc.)
+- `/lib/` - Nix library functions
+  - `dns-zone.nix` - DNS zone generation functions
 - `/services/` - Reusable service modules, selectively imported by hosts
   - `home-assistant/` - Home automation stack
   - `monitoring/` - Observability stack (Prometheus, Grafana, Loki, Tempo)
-  - `ns/` - DNS services (authoritative, resolver)
+  - `ns/` - DNS services (authoritative, resolver, zone generation)
   - `http-proxy/`, `ca/`, `postgres/`, `nats/`, `jellyfin/`, etc.
 - `/secrets/` - SOPS-encrypted secrets with age encryption
 - `/common/` - Shared configurations (e.g., VM guest agent)
@@ -137,6 +147,7 @@ All hosts automatically get:
 - Daily auto-upgrades with auto-reboot
 - Prometheus node-exporter + Promtail (logs to monitoring01)
 - Custom root CA trust
+- DNS zone auto-registration via `homelab.dns` options
 
 ### Active Hosts
 
@@ -279,14 +290,17 @@ This means:
 1. Create `/hosts/<hostname>/` directory
 2. Copy structure from `template1` or similar host
 3. Add host entry to `flake.nix` nixosConfigurations
-4. Add hostname to dns zone files. Merge to master. Run auto-upgrade on dns servers.
+4. Configure networking in `configuration.nix` (static IP via `systemd.network.networks`, DNS servers)
-5. User clones template host
+5. (Optional) Add `homelab.dns.cnames` if the host needs CNAME aliases
-6. User runs `prepare-host.sh` on new host, this deletes files which should be regenerated, like ssh host keys, machine-id etc. It also creates a new age key, and prints the public key
+6. User clones template host
-7. This key is then added to `.sops.yaml`
+7. User runs `prepare-host.sh` on new host, this deletes files which should be regenerated, like ssh host keys, machine-id etc. It also creates a new age key, and prints the public key
-8. Create `/secrets/<hostname>/` if needed
+8. This key is then added to `.sops.yaml`
-9. Configure networking (static IP, DNS servers)
+9. Create `/secrets/<hostname>/` if needed
 10. Commit changes, and merge to master.
 11. Deploy by running `nixos-rebuild boot --flake URL#<hostname>` on the host.
+12. Run auto-upgrade on DNS servers (ns1, ns2) to pick up the new host's DNS entry
+
+**Note:** DNS A records are auto-generated from the host's `systemd.network.networks` static IP configuration. No manual zone file editing is required.
 
 ### Important Patterns
 
@@ -313,5 +327,24 @@ All hosts ship metrics and logs to `monitoring01`:
 
 - `ns1` (10.69.13.5) - Primary authoritative DNS + resolver
 - `ns2` (10.69.13.6) - Secondary authoritative DNS (AXFR from ns1)
-- Zone files managed in `/services/ns/`
 - All hosts point to ns1/ns2 for DNS resolution
+
+**Zone Auto-Generation:**
+
+DNS zone entries are automatically generated from host configurations:
+
+- **Flake-managed hosts**: A records extracted from `systemd.network.networks` static IPs
+- **CNAMEs**: Defined via `homelab.dns.cnames` option in host configs
+- **External hosts**: Non-flake hosts defined in `/services/ns/external-hosts.nix`
+- **Serial number**: Uses `self.sourceInfo.lastModified` (git commit timestamp)
+
+Host DNS options (`homelab.dns.*`):
+- `enable` (default: `true`) - Include host in DNS zone generation
+- `cnames` (default: `[]`) - List of CNAME aliases pointing to this host
+
+Hosts are automatically excluded from DNS if:
+- `homelab.dns.enable = false` (e.g., template hosts)
+- No static IP configured (e.g., DHCP-only hosts)
+- Network interface is a VPN/tunnel (wg*, tun*, tap*)
+
+To add DNS entries for non-NixOS hosts, edit `/services/ns/external-hosts.nix`.

hosts/auth01/configuration.nix

@@ -11,6 +11,8 @@
     ../../common/vm
   ];
 
+  homelab.dns.cnames = [ "ldap" ];
+
   nixpkgs.config.allowUnfree = true;
   # Use the systemd-boot EFI boot loader.
   boot.loader.grub = {

hosts/http-proxy/configuration.nix

@@ -11,6 +11,22 @@
     ../../common/vm
   ];
 
+  homelab.dns.cnames = [
+    "nzbget"
+    "radarr"
+    "sonarr"
+    "ha"
+    "z2m"
+    "grafana"
+    "prometheus"
+    "alertmanager"
+    "jelly"
+    "auth"
+    "lldap"
+    "pyroscope"
+    "pushgw"
+  ];
+
   nixpkgs.config.allowUnfree = true;
   # Use the systemd-boot EFI boot loader.
   boot.loader.grub = {

hosts/nix-cache01/configuration.nix

@@ -11,6 +11,8 @@
     ../../common/vm
   ];
 
+  homelab.dns.cnames = [ "nix-cache" "actions1" ];
+
   fileSystems."/nix" = {
     device = "/dev/disk/by-label/nixcache";
     fsType = "xfs";

hosts/template/configuration.nix

@@ -8,6 +8,9 @@
       ../../system
     ];
 
+  # Template host - exclude from DNS zone generation
+  homelab.dns.enable = false;
+
 
   boot.loader.grub.enable = true;
   boot.loader.grub.device = "/dev/sda";

hosts/testvm01/configuration.nix

@@ -13,6 +13,9 @@
     ../../common/vm
   ];
 
+  # Test VM - exclude from DNS zone generation
+  homelab.dns.enable = false;
+
   nixpkgs.config.allowUnfree = true;
   boot.loader.grub.enable = true;
   boot.loader.grub.device = "/dev/vda";

hosts/vault01/configuration.nix

@@ -14,6 +14,8 @@
     ../../services/vault
   ];
 
+  homelab.dns.cnames = [ "vault" ];
+
   nixpkgs.config.allowUnfree = true;
   boot.loader.grub.enable = true;
   boot.loader.grub.device = "/dev/vda";

lib/dns-zone.nix

@@ -0,0 +1,160 @@
+{ lib }:
+let
+  # Pad string on the right to reach a fixed width
+  rightPad = width: str:
+    let
+      len = builtins.stringLength str;
+      padding = if len >= width then "" else lib.strings.replicate (width - len) " ";
+    in
+    str + padding;
+
+  # Extract IP address from CIDR notation (e.g., "10.69.13.5/24" -> "10.69.13.5")
+  extractIP = address:
+    let
+      parts = lib.splitString "/" address;
+    in
+    builtins.head parts;
+
+  # Check if a network interface name looks like a VPN/tunnel interface
+  isVpnInterface = ifaceName:
+    lib.hasPrefix "wg" ifaceName ||
+    lib.hasPrefix "tun" ifaceName ||
+    lib.hasPrefix "tap" ifaceName ||
+    lib.hasPrefix "vti" ifaceName;
+
+  # Extract DNS information from a single host configuration
+  # Returns null if host should not be included in DNS
+  extractHostDNS = name: hostConfig:
+    let
+      cfg = hostConfig.config;
+      # Handle cases where homelab module might not be imported
+      dnsConfig = (cfg.homelab or { }).dns or { enable = true; cnames = [ ]; };
+      hostname = cfg.networking.hostName;
+      networks = cfg.systemd.network.networks or { };
+
+      # Filter out VPN interfaces and find networks with static addresses
+      # Check matchConfig.Name instead of network unit name (which can have prefixes like "40-")
+      physicalNetworks = lib.filterAttrs
+        (netName: netCfg:
+          let
+            ifaceName = netCfg.matchConfig.Name or "";
+          in
+          !(isVpnInterface ifaceName) && (netCfg.address or [ ]) != [ ])
+        networks;
+
+      # Get addresses from physical networks only
+      networkAddresses = lib.flatten (
+        lib.mapAttrsToList
+          (netName: netCfg: netCfg.address or [ ])
+          physicalNetworks
+      );
+
+      # Get the first address, if any
+      firstAddress = if networkAddresses != [ ] then builtins.head networkAddresses else null;
+
+      # Check if host uses DHCP (no static address)
+      usesDHCP = firstAddress == null ||
+        lib.any
+          (netName: (networks.${netName}.networkConfig.DHCP or "no") != "no")
+          (lib.attrNames networks);
+    in
+    if !(dnsConfig.enable or true) || firstAddress == null then
+      null
+    else
+      {
+        inherit hostname;
+        ip = extractIP firstAddress;
+        cnames = dnsConfig.cnames or [ ];
+      };
+
+  # Generate A record line
+  generateARecord = hostname: ip:
+    "${rightPad 20 hostname}IN      A       ${ip}";
+
+  # Generate CNAME record line
+  generateCNAME = alias: target:
+    "${rightPad 20 alias}IN      CNAME   ${target}";
+
+  # Generate zone file from flake configurations and external hosts
+  generateZone =
+    { self
+    , externalHosts
+    , serial
+    , domain ? "home.2rjus.net"
+    , ttl ? 1800
+    , refresh ? 3600
+    , retry ? 900
+    , expire ? 1209600
+    , minTtl ? 120
+    , nameservers ? [ "ns1" "ns2" "ns3" ]
+    , adminEmail ? "admin.test.2rjus.net"
+    }:
+    let
+      # Extract DNS info from all flake hosts
+      nixosConfigs = self.nixosConfigurations or { };
+      hostDNSList = lib.filter (x: x != null) (
+        lib.mapAttrsToList extractHostDNS nixosConfigs
+      );
+
+      # Sort hosts by IP for consistent output
+      sortedHosts = lib.sort (a: b: a.ip < b.ip) hostDNSList;
+
+      # Generate A records for flake hosts
+      flakeARecords = lib.concatMapStringsSep "\n" (host:
+        generateARecord host.hostname host.ip
+      ) sortedHosts;
+
+      # Generate CNAMEs for flake hosts
+      flakeCNAMEs = lib.concatMapStringsSep "\n" (host:
+        lib.concatMapStringsSep "\n" (cname:
+          generateCNAME cname host.hostname
+        ) host.cnames
+      ) (lib.filter (h: h.cnames != [ ]) sortedHosts);
+
+      # Generate A records for external hosts
+      externalARecords = lib.concatStringsSep "\n" (
+        lib.mapAttrsToList (name: ip:
+          generateARecord name ip
+        ) (externalHosts.aRecords or { })
+      );
+
+      # Generate CNAMEs for external hosts
+      externalCNAMEs = lib.concatStringsSep "\n" (
+        lib.mapAttrsToList (alias: target:
+          generateCNAME alias target
+        ) (externalHosts.cnames or { })
+      );
+
+      # NS records
+      nsRecords = lib.concatMapStringsSep "\n" (ns:
+        "                    IN      NS      ${ns}.${domain}."
+      ) nameservers;
+
+      # SOA record
+      soa = ''
+        $ORIGIN ${domain}.
+        $TTL ${toString ttl}
+        @       IN      SOA     ns1.${domain}.      ${adminEmail}. (
+                                ${toString serial}                    ; serial number
+                                ${toString refresh}                    ; refresh
+                                ${toString retry}                     ; retry
+                                ${toString expire}                 ; expire
+                                ${toString minTtl}                     ; ttl
+                                )'';
+    in
+    lib.concatStringsSep "\n\n" (lib.filter (s: s != "") [
+      soa
+      nsRecords
+      "; Flake-managed hosts (auto-generated)"
+      flakeARecords
+      (if flakeCNAMEs != "" then "; Flake-managed CNAMEs\n${flakeCNAMEs}" else "")
+      "; External hosts (not managed by this flake)"
+      externalARecords
+      (if externalCNAMEs != "" then "; External CNAMEs\n${externalCNAMEs}" else "")
+      ""
+    ]);
+
+in
+{
+  inherit extractIP extractHostDNS generateARecord generateCNAME generateZone;
+}

modules/homelab/default.nix

@@ -0,0 +1,6 @@
+{ ... }:
+{
+  imports = [
+    ./dns.nix
+  ];
+}

modules/homelab/dns.nix

@@ -0,0 +1,20 @@
+{ config, lib, ... }:
+let
+  cfg = config.homelab.dns;
+in
+{
+  options.homelab.dns = {
+    enable = lib.mkOption {
+      type = lib.types.bool;
+      default = true;
+      description = "Include this host in DNS zone generation";
+    };
+
+    cnames = lib.mkOption {
+      type = lib.types.listOf lib.types.str;
+      default = [ ];
+      description = "CNAME records pointing to this host";
+      example = [ "web" "api" ];
+    };
+  };
+}

services/ns/external-hosts.nix

@@ -0,0 +1,33 @@
+# DNS records for hosts not managed by this flake
+# These are manually maintained and combined with auto-generated records
+{
+  aRecords = {
+    # 10
+    "gw" = "10.69.10.1";
+
+    # 12_CORE
+    "nas" = "10.69.12.50";
+    "nzbget-jail" = "10.69.12.51";
+    "restic" = "10.69.12.52";
+    "radarr-jail" = "10.69.12.53";
+    "sonarr-jail" = "10.69.12.54";
+    "bazarr" = "10.69.12.55";
+    "pve1" = "10.69.12.75";
+    "inc1" = "10.69.12.80";
+
+    # 22_WLAN
+    "unifi-ctrl" = "10.69.22.5";
+
+    # 30
+    "gunter" = "10.69.30.105";
+
+    # 31
+    "media" = "10.69.31.50";
+
+    # 99_MGMT
+    "sw1" = "10.69.99.2";
+  };
+
+  cnames = {
+  };
+}

services/ns/master-authorative.nix

@@ -1,4 +1,16 @@
-{ ... }:
+{ self, lib, ... }:
+let
+  dnsLib = import ../../lib/dns-zone.nix { inherit lib; };
+  externalHosts = import ./external-hosts.nix;
+
+  # Generate zone from flake hosts + external hosts
+  # Use lastModified from git commit as serial number
+  zoneData = dnsLib.generateZone {
+    inherit self externalHosts;
+    serial = self.sourceInfo.lastModified;
+    domain = "home.2rjus.net";
+  };
+in
 {
   sops.secrets.ns_xfer_key = {
     path = "/etc/nsd/xfer.key";
@@ -26,7 +38,7 @@
       "home.2rjus.net" = {
         provideXFR = [ "10.69.13.6 xferkey" ];
         notify = [ "10.69.13.6@8053 xferkey" ];
-        data = builtins.readFile ./zones-home-2rjus-net.conf;
+        data = zoneData;
       };
     };
   };

services/ns/secondary-authorative.nix

@@ -1,4 +1,16 @@
-{ ... }:
+{ self, lib, ... }:
+let
+  dnsLib = import ../../lib/dns-zone.nix { inherit lib; };
+  externalHosts = import ./external-hosts.nix;
+
+  # Generate zone from flake hosts + external hosts
+  # Used as initial zone data before first AXFR completes
+  zoneData = dnsLib.generateZone {
+    inherit self externalHosts;
+    serial = self.sourceInfo.lastModified;
+    domain = "home.2rjus.net";
+  };
+in
 {
   sops.secrets.ns_xfer_key = {
     path = "/etc/nsd/xfer.key";
@@ -24,7 +36,7 @@
       "home.2rjus.net" = {
         allowNotify = [ "10.69.13.5 xferkey" ];
         requestXFR = [ "AXFR 10.69.13.5@8053 xferkey" ];
-        data = builtins.readFile ./zones-home-2rjus-net.conf;
+        data = zoneData;
       };
     };
   };

services/ns/zones-home-2rjus-net.conf

@@ -1,99 +0,0 @@
-$ORIGIN home.2rjus.net.
-$TTL 1800
-@       IN      SOA     ns1.home.2rjus.net.      admin.test.2rjus.net. (
-                        2066                    ; serial number
-                        3600                    ; refresh
-                        900                     ; retry
-                        1209600                 ; expire
-                        120                     ; ttl
-                        )
-
-                    IN      NS      ns1.home.2rjus.net.
-                    IN      NS      ns2.home.2rjus.net.
-                    IN      NS      ns3.home.2rjus.net.
-
-; 8_k8s
-kube-blue1          IN      A       10.69.8.150
-kube-blue2          IN      A       10.69.8.151
-kube-blue3          IN      A       10.69.8.152
-
-kube-blue4          IN      A       10.69.8.153
-rook                IN      CNAME   kube-blue4
-
-kube-blue5          IN      A       10.69.8.154
-git                 IN      CNAME   kube-blue5
-
-kube-blue6          IN      A       10.69.8.155
-kube-blue7          IN      A       10.69.8.156
-kube-blue8          IN      A       10.69.8.157
-kube-blue9          IN      A       10.69.8.158
-kube-blue10         IN      A       10.69.8.159
-
-; 10
-gw                  IN      A       10.69.10.1
-
-; 12_CORE
-virt-mini1          IN      A       10.69.12.11
-nas                 IN      A       10.69.12.50
-nzbget-jail         IN      A       10.69.12.51
-restic              IN      A       10.69.12.52
-radarr-jail         IN      A       10.69.12.53
-sonarr-jail         IN      A       10.69.12.54
-bazarr              IN      A       10.69.12.55
-mpnzb               IN      A       10.69.12.57
-pve1                IN      A       10.69.12.75
-inc1                IN      A       10.69.12.80
-inc2                IN      A       10.69.12.81
-media1              IN      A       10.69.12.82
-
-; 13_SVC
-ns1                 IN      A       10.69.13.5
-ns2                 IN      A       10.69.13.6
-ns3                 IN      A       10.69.13.7
-ns4                 IN      A       10.69.13.8
-ha1                 IN      A       10.69.13.9
-nixos-test1         IN      A       10.69.13.10
-http-proxy          IN      A       10.69.13.11
-ca                  IN      A       10.69.13.12
-monitoring01        IN      A       10.69.13.13
-jelly01             IN      A       10.69.13.14
-nix-cache01         IN      A       10.69.13.15
-nix-cache           IN      CNAME   nix-cache01
-actions1            IN      CNAME   nix-cache01
-pgdb1               IN      A       10.69.13.16
-nats1               IN      A       10.69.13.17
-auth01              IN      A       10.69.13.18
-vault01             IN      A       10.69.13.19
-vault               IN      CNAME   vault01
-vaulttest01         IN      A       10.69.13.150
-
-; http-proxy cnames
-nzbget              IN      CNAME   http-proxy
-radarr              IN      CNAME   http-proxy
-sonarr              IN      CNAME   http-proxy
-ha                  IN      CNAME   http-proxy
-z2m                 IN      CNAME   http-proxy
-grafana             IN      CNAME   http-proxy
-prometheus          IN      CNAME   http-proxy
-alertmanager        IN      CNAME   http-proxy
-jelly               IN      CNAME   http-proxy
-auth                IN      CNAME   http-proxy
-lldap               IN      CNAME   http-proxy
-pyroscope           IN      CNAME   http-proxy
-pushgw              IN      CNAME   http-proxy
-
-ldap                IN      CNAME   auth01
-
-
-; 22_WLAN
-unifi-ctrl          IN      A       10.69.22.5
-
-; 30
-gunter              IN      A       10.69.30.105
-
-; 31
-media               IN      A       10.69.31.50
-
-; 99_MGMT
-sw1                 IN      A       10.69.99.2
-testing             IN      A       10.69.33.33

system/default.nix

@@ -11,5 +11,7 @@
     ./sops.nix
     ./sshd.nix
     ./vault-secrets.nix
+
+    ../modules/homelab
   ];
 }