pipeline: add testing improvements for branch-based workflows

Implement dual improvements to enable efficient testing of pipeline changes
without polluting master branch:

1. Add --force flag to create-host script
   - Skip hostname/IP uniqueness validation
   - Overwrite existing host configurations
   - Update entries in flake.nix and terraform/vms.tf (no duplicates)
   - Useful for iterating on configurations during testing

2. Add branch support to bootstrap mechanism
   - Bootstrap service reads NIXOS_FLAKE_BRANCH environment variable
   - Defaults to master if not set
   - Uses branch in git URL via ?ref= parameter
   - Service loads environment from /etc/environment

3. Add cloud-init disk support for branch configuration
   - VMs can specify flake_branch field in terraform/vms.tf
   - Automatically generates cloud-init snippet setting NIXOS_FLAKE_BRANCH
   - Uploads snippet to Proxmox via SSH
   - Production VMs omit flake_branch and use master

4. Update documentation
   - Document --force flag usage in create-host README
   - Add branch testing examples in terraform README
   - Update TODO.md with testing workflow
   - Add .generated/ to gitignore

Testing workflow: Create feature branch, set flake_branch in VM definition,
deploy with terraform, iterate with --force flag, clean up before merging.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

TODO.md

@@ -54,6 +54,7 @@ Automate the entire process of creating, configuring, and deploying new NixOS ho
 
 **Status:** ✅ Fully implemented and tested
 **Completed:** 2025-02-01
+**Enhanced:** 2025-02-01 (added --force flag)
 
 **Goal:** Automate creation of host configuration files
 
@@ -64,6 +65,7 @@ Automate the entire process of creating, configuring, and deploying new NixOS ho
 - Comprehensive validation (hostname format/uniqueness, IP subnet/uniqueness)
 - Jinja2 templates for NixOS configurations
 - Automatic updates to flake.nix and terraform/vms.tf
+- `--force` flag for regenerating existing configurations (useful for testing)
 
 **Tasks:**
 - [x] Create Python CLI with typer framework
@@ -109,6 +111,7 @@ create-host \
 
 **Status:** ✅ Fully implemented and tested
 **Completed:** 2025-02-01
+**Enhanced:** 2025-02-01 (added branch support for testing)
 
 **Goal:** Get freshly deployed VM to apply its specific host configuration
 
@@ -118,7 +121,8 @@ create-host \
 - Systemd service `nixos-bootstrap.service` runs on first boot
 - Depends on `cloud-config.service` to ensure hostname is set
 - Reads hostname from `hostnamectl` (set by cloud-init via Terraform)
-- Runs `nixos-rebuild boot --flake git+https://git.t-juice.club/torjus/nixos-servers.git#${hostname}`
+- Supports custom git branch via `NIXOS_FLAKE_BRANCH` environment variable
+- Runs `nixos-rebuild boot --flake git+https://git.t-juice.club/torjus/nixos-servers.git?ref=$BRANCH#${hostname}`
 - Reboots into new configuration on success
 - Fails gracefully without reboot on errors (network issues, missing config)
 - Service self-destructs after successful bootstrap (not in new config)
@@ -240,10 +244,80 @@ Since most hosts use static IPs defined in their NixOS configurations, we can ex
 
 ### Phase 7: Testing & Documentation
 
-**Tasks:**
-- [ ] Test full pipeline end-to-end
-- [ ] Create test host and verify all steps
-- [ ] Document the new workflow in CLAUDE.md
+**Status:** 🚧 In Progress (testing improvements completed)
+
+**Testing Improvements Implemented (2025-02-01):**
+
+The pipeline now supports efficient testing without polluting master branch:
+
+**1. --force Flag for create-host**
+- Re-run `create-host` to regenerate existing configurations
+- Updates existing entries in flake.nix and terraform/vms.tf (no duplicates)
+- Skip uniqueness validation checks
+- Useful for iterating on configuration templates during testing
+
+**2. Branch Support for Bootstrap**
+- Bootstrap service reads `NIXOS_FLAKE_BRANCH` environment variable
+- Defaults to `master` if not set
+- Allows testing pipeline changes on feature branches
+- Cloud-init passes branch via `/etc/environment`
+
+**3. Cloud-init Disk for Branch Configuration**
+- Terraform generates custom cloud-init snippets for test VMs
+- Set `flake_branch` field in VM definition to use non-master branch
+- Production VMs omit this field and use master (default)
+- Files automatically uploaded to Proxmox via SSH
+
+**Testing Workflow:**
+
+```bash
+# 1. Create test branch
+git checkout -b test-pipeline
+
+# 2. Generate or update host config
+create-host --hostname testvm01 --ip 10.69.13.100/24
+
+# 3. Edit terraform/vms.tf to add test VM with branch
+# vms = {
+#   "testvm01" = {
+#     ip = "10.69.13.100/24"
+#     flake_branch = "test-pipeline"  # Bootstrap from this branch
+#   }
+# }
+
+# 4. Commit and push test branch
+git add -A && git commit -m "test: add testvm01"
+git push origin test-pipeline
+
+# 5. Deploy VM
+cd terraform && tofu apply
+
+# 6. Watch bootstrap (VM fetches from test-pipeline branch)
+ssh root@10.69.13.100
+journalctl -fu nixos-bootstrap.service
+
+# 7. Iterate: modify templates and regenerate with --force
+cd .. && create-host --hostname testvm01 --ip 10.69.13.100/24 --force
+git commit -am "test: update config" && git push
+
+# Redeploy to test fresh bootstrap
+cd terraform
+tofu destroy -target=proxmox_vm_qemu.vm[\"testvm01\"] && tofu apply
+
+# 8. Clean up when done: squash commits, merge to master, remove test VM
+```
+
+**Files:**
+- `scripts/create-host/create_host.py` - Added --force parameter
+- `scripts/create-host/manipulators.py` - Update vs insert logic
+- `hosts/template2/bootstrap.nix` - Branch support via environment variable
+- `terraform/vms.tf` - flake_branch field support
+- `terraform/cloud-init.tf` - Custom cloud-init disk generation
+- `terraform/variables.tf` - proxmox_host variable for SSH uploads
+
+**Remaining Tasks:**
+- [ ] Test full pipeline end-to-end on feature branch
+- [ ] Update CLAUDE.md with testing workflow
 - [ ] Add troubleshooting section
 - [ ] Create examples for common scenarios (DHCP host, static IP host, etc.)