Torjus Håkestad
83de9a3ffb
Some checks failed
Run nix flake check / flake-check (push) Has been cancelled
Implement dual improvements to enable efficient testing of pipeline changes without polluting master branch: 1. Add --force flag to create-host script - Skip hostname/IP uniqueness validation - Overwrite existing host configurations - Update entries in flake.nix and terraform/vms.tf (no duplicates) - Useful for iterating on configurations during testing 2. Add branch support to bootstrap mechanism - Bootstrap service reads NIXOS_FLAKE_BRANCH environment variable - Defaults to master if not set - Uses branch in git URL via ?ref= parameter - Service loads environment from /etc/environment 3. Add cloud-init disk support for branch configuration - VMs can specify flake_branch field in terraform/vms.tf - Automatically generates cloud-init snippet setting NIXOS_FLAKE_BRANCH - Uploads snippet to Proxmox via SSH - Production VMs omit flake_branch and use master 4. Update documentation - Document --force flag usage in create-host README - Add branch testing examples in terraform README - Update TODO.md with testing workflow - Add .generated/ to gitignore Testing workflow: Create feature branch, set flake_branch in VM definition, deploy with terraform, iterate with --force flag, clean up before merging. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
269 lines
6.9 KiB
Markdown
269 lines
6.9 KiB
Markdown
# NixOS Host Configuration Generator
|
|
|
|
Automated tool for generating NixOS host configurations, flake.nix entries, and Terraform VM definitions for homelab infrastructure.
|
|
|
|
## Installation
|
|
|
|
The tool is available in the Nix development shell:
|
|
|
|
```bash
|
|
nix develop
|
|
```
|
|
|
|
## Usage
|
|
|
|
### Basic Usage
|
|
|
|
Create a new host with DHCP networking:
|
|
|
|
```bash
|
|
python -m scripts.create_host.create_host create --hostname test01
|
|
```
|
|
|
|
Create a new host with static IP:
|
|
|
|
```bash
|
|
python -m scripts.create_host.create_host create \
|
|
--hostname test01 \
|
|
--ip 10.69.13.50/24
|
|
```
|
|
|
|
Create a host with custom resources:
|
|
|
|
```bash
|
|
python -m scripts.create_host.create_host create \
|
|
--hostname bighost01 \
|
|
--ip 10.69.13.51/24 \
|
|
--cpu 8 \
|
|
--memory 8192 \
|
|
--disk 100G
|
|
```
|
|
|
|
### Dry Run Mode
|
|
|
|
Preview what would be created without making changes:
|
|
|
|
```bash
|
|
python -m scripts.create_host.create_host create \
|
|
--hostname test01 \
|
|
--ip 10.69.13.50/24 \
|
|
--dry-run
|
|
```
|
|
|
|
### Force Mode (Regenerate Existing Configuration)
|
|
|
|
Overwrite an existing host configuration (useful for testing):
|
|
|
|
```bash
|
|
python -m scripts.create_host.create_host create \
|
|
--hostname test01 \
|
|
--ip 10.69.13.50/24 \
|
|
--force
|
|
```
|
|
|
|
This mode:
|
|
- Skips hostname and IP uniqueness validation
|
|
- Overwrites files in `hosts/<hostname>/`
|
|
- Updates existing entries in `flake.nix` and `terraform/vms.tf` (doesn't duplicate)
|
|
- Useful for iterating on configuration templates during testing
|
|
|
|
### Options
|
|
|
|
- `--hostname` (required): Hostname for the new host
|
|
- Must be lowercase alphanumeric with hyphens
|
|
- Must be unique (not already exist in repository)
|
|
|
|
- `--ip` (optional): Static IP address with CIDR notation
|
|
- Format: `10.69.13.X/24`
|
|
- Must be in `10.69.13.0/24` subnet
|
|
- Last octet must be 1-254
|
|
- Omit this option for DHCP configuration
|
|
|
|
- `--cpu` (optional, default: 2): Number of CPU cores
|
|
- Must be at least 1
|
|
|
|
- `--memory` (optional, default: 2048): Memory in MB
|
|
- Must be at least 512
|
|
|
|
- `--disk` (optional, default: "20G"): Disk size
|
|
- Examples: "20G", "50G", "100G"
|
|
|
|
- `--dry-run` (flag): Preview changes without creating files
|
|
|
|
- `--force` (flag): Overwrite existing host configuration
|
|
- Skips uniqueness validation
|
|
- Updates existing entries instead of creating duplicates
|
|
|
|
## What It Does
|
|
|
|
The tool performs the following actions:
|
|
|
|
1. **Validates** the configuration:
|
|
- Hostname format (RFC 1123 compliance)
|
|
- Hostname uniqueness
|
|
- IP address format and subnet (if provided)
|
|
- IP address uniqueness (if provided)
|
|
|
|
2. **Generates** host configuration files:
|
|
- `hosts/<hostname>/default.nix` - Import wrapper
|
|
- `hosts/<hostname>/configuration.nix` - Full host configuration
|
|
|
|
3. **Updates** repository files:
|
|
- `flake.nix` - Adds new nixosConfigurations entry
|
|
- `terraform/vms.tf` - Adds new VM definition
|
|
|
|
4. **Displays** next steps for:
|
|
- Reviewing changes with git diff
|
|
- Verifying NixOS configuration
|
|
- Verifying Terraform configuration
|
|
- Committing changes
|
|
- Deploying the VM
|
|
|
|
## Generated Configuration
|
|
|
|
### Host Features
|
|
|
|
All generated hosts include:
|
|
|
|
- Full system imports from `../../system`:
|
|
- Nix binary cache integration
|
|
- SSH with root login
|
|
- SOPS secrets management
|
|
- Internal ACME CA integration
|
|
- Daily auto-upgrades with auto-reboot
|
|
- Prometheus node-exporter
|
|
- Promtail logging to monitoring01
|
|
|
|
- VM guest agent from `../../common/vm`
|
|
- Hardware configuration from `../template/hardware-configuration.nix`
|
|
|
|
### Networking
|
|
|
|
**Static IP mode** (when `--ip` is provided):
|
|
```nix
|
|
systemd.network.networks."ens18" = {
|
|
matchConfig.Name = "ens18";
|
|
address = [ "10.69.13.50/24" ];
|
|
routes = [ { Gateway = "10.69.13.1"; } ];
|
|
linkConfig.RequiredForOnline = "routable";
|
|
};
|
|
```
|
|
|
|
**DHCP mode** (when `--ip` is omitted):
|
|
```nix
|
|
systemd.network.networks."ens18" = {
|
|
matchConfig.Name = "ens18";
|
|
networkConfig.DHCP = "ipv4";
|
|
linkConfig.RequiredForOnline = "routable";
|
|
};
|
|
```
|
|
|
|
### DNS Configuration
|
|
|
|
All hosts are configured with:
|
|
- DNS servers: `10.69.13.5`, `10.69.13.6` (ns1, ns2)
|
|
- Domain: `home.2rjus.net`
|
|
|
|
## Examples
|
|
|
|
### Create a test VM with defaults
|
|
|
|
```bash
|
|
python -m scripts.create_host.create_host create --hostname test99
|
|
```
|
|
|
|
This creates a DHCP VM with 2 CPU cores, 2048 MB memory, and 20G disk.
|
|
|
|
### Create a database server with static IP
|
|
|
|
```bash
|
|
python -m scripts.create_host.create_host create \
|
|
--hostname pgdb2 \
|
|
--ip 10.69.13.52/24 \
|
|
--cpu 4 \
|
|
--memory 4096 \
|
|
--disk 50G
|
|
```
|
|
|
|
### Preview changes before creating
|
|
|
|
```bash
|
|
python -m scripts.create_host.create_host create \
|
|
--hostname test99 \
|
|
--ip 10.69.13.99/24 \
|
|
--dry-run
|
|
```
|
|
|
|
## Error Handling
|
|
|
|
The tool validates input and provides clear error messages for:
|
|
|
|
- Invalid hostname format (must be lowercase alphanumeric with hyphens)
|
|
- Duplicate hostname (already exists in repository)
|
|
- Invalid IP format (must be X.X.X.X/24)
|
|
- Wrong subnet (must be 10.69.13.0/24)
|
|
- Invalid last octet (must be 1-254)
|
|
- Duplicate IP address (already in use)
|
|
- Resource constraints (CPU < 1, memory < 512 MB)
|
|
|
|
## Integration with Deployment Pipeline
|
|
|
|
This tool implements **Phase 2** of the automated deployment pipeline:
|
|
|
|
1. **Phase 1**: Template building ✓ (build-and-deploy-template.yml)
|
|
2. **Phase 2**: Host configuration generation ✓ (this tool)
|
|
3. **Phase 3**: Bootstrap automation (planned)
|
|
4. **Phase 4**: Secrets management (planned)
|
|
5. **Phase 5**: DNS automation (planned)
|
|
6. **Phase 6**: Full integration (planned)
|
|
|
|
## Development
|
|
|
|
### Project Structure
|
|
|
|
```
|
|
scripts/create-host/
|
|
├── create_host.py # Main CLI entry point (typer app)
|
|
├── __init__.py # Package initialization
|
|
├── validators.py # Validation logic
|
|
├── generators.py # File generation using Jinja2
|
|
├── manipulators.py # Text manipulation for flake.nix and vms.tf
|
|
├── models.py # Data models (HostConfig)
|
|
├── templates/
|
|
│ ├── default.nix.j2 # Template for default.nix
|
|
│ └── configuration.nix.j2 # Template for configuration.nix
|
|
└── README.md # This file
|
|
```
|
|
|
|
### Testing
|
|
|
|
Run the test cases from the implementation plan:
|
|
|
|
```bash
|
|
# Test 1: DHCP host with defaults
|
|
python -m scripts.create_host.create_host create --hostname testdhcp --dry-run
|
|
|
|
# Test 2: Static IP host
|
|
python -m scripts.create_host.create_host create \
|
|
--hostname test50 --ip 10.69.13.50/24 --dry-run
|
|
|
|
# Test 3: Custom resources
|
|
python -m scripts.create_host.create_host create \
|
|
--hostname test51 --ip 10.69.13.51/24 \
|
|
--cpu 8 --memory 8192 --disk 100G --dry-run
|
|
|
|
# Test 4: Duplicate hostname (should error)
|
|
python -m scripts.create_host.create_host create --hostname ns1 --dry-run
|
|
|
|
# Test 5: Invalid subnet (should error)
|
|
python -m scripts.create_host.create_host create \
|
|
--hostname testbad --ip 192.168.1.50/24 --dry-run
|
|
|
|
# Test 6: Invalid hostname (should error)
|
|
python -m scripts.create_host.create_host create --hostname Test_Host --dry-run
|
|
```
|
|
|
|
## License
|
|
|
|
Part of the nixos-servers homelab infrastructure repository.
|