docs: finalize remote access plan with WireGuard gateway design

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

docs/plans/remote-access.md

@@ -4,119 +4,127 @@
 
 ## Goal
 
-Enable remote access to some or all homelab services from outside the internal network, without exposing anything directly to the internet.
+Enable personal remote access to selected homelab services from outside the internal network, without exposing anything directly to the internet.
 
 ## Current State
 
 - All services are only accessible from the internal 10.69.13.x network
-- Exception: jelly01 has a WireGuard link to an external VPS
+- http-proxy has a WireGuard tunnel (`wg0`, `10.69.222.0/24`) to a VPS (`docker2.t-juice.club`) on an OpenStack cluster
-- No services are directly exposed to the public internet
+- VPS runs Traefik which proxies selected services (including Jellyfin) back through the tunnel to http-proxy's Caddy
+- No other services are directly exposed to the public internet
 
-## Constraints
+## Decision: WireGuard Gateway
 
-- Nothing should be directly accessible from the outside
+After evaluating WireGuard gateway vs Headscale (self-hosted Tailscale), the **WireGuard gateway** approach was chosen:
-- Must use VPN or overlay network (no port forwarding of services)
-- Self-hosted solutions preferred over managed services
 
-## Options
+- Only 2 client devices (laptop + phone), so Headscale's device management UX isn't needed
+- Split DNS works fine on Linux laptop via systemd-resolved; all-or-nothing DNS on phone is acceptable for occasional use
+- Simpler infrastructure - no control server to maintain
+- Builds on existing WireGuard experience and setup
 
-### 1. WireGuard Gateway (Internal Router)
+## Architecture
 
-A dedicated NixOS host on the internal network with a WireGuard tunnel out to the VPS. The VPS becomes the public entry point, and the gateway routes traffic to internal services. Firewall rules on the gateway control which services are reachable.
+```
+                    ┌─────────────────────────────────┐
+                    │  VPS (OpenStack)                │
+  Laptop/Phone ──→ │  WireGuard endpoint             │
+  (WireGuard)      │  Client peers: laptop, phone    │
+                    │  Routes 10.69.13.0/24 via tunnel│
+                    └──────────┬──────────────────────┘
+                               │ WireGuard tunnel
+                               ▼
+                    ┌─────────────────────────────────┐
+                    │  extgw01 (gateway + bastion)    │
+                    │  - WireGuard tunnel to VPS      │
+                    │  - Firewall (allowlist only)    │
+                    │  - SSH + 2FA (full access)      │
+                    └──────────┬──────────────────────┘
+                               │ allowed traffic only
+                               ▼
+                    ┌─────────────────────────────────┐
+                    │  Internal network 10.69.13.0/24 │
+                    │  - monitoring01:3000 (Grafana)  │
+                    │  - jelly01:8096 (Jellyfin)      │
+                    │  - *-jail hosts (arr stack)     │
+                    └─────────────────────────────────┘
+```
 
-**Pros:**
+### Existing path (unchanged)
-- Simple, well-understood technology
-- Already running WireGuard for jelly01
-- Full control over routing and firewall rules
-- Excellent NixOS module support
-- No extra dependencies
 
-**Cons:**
+The current public access path stays as-is:
-- Hub-and-spoke topology (all traffic goes through VPS)
-- Manual peer management
-- Adding a new client device means editing configs on both VPS and gateway
 
-### 2. WireGuard Mesh (No Relay)
+```
+Internet → VPS (Traefik) → WireGuard → http-proxy (Caddy) → internal services
+```
 
-Each client device connects directly to a WireGuard endpoint. Could be on the VPS which forwards to the homelab, or if there is a routable IP at home, directly to an internal host.
+This handles public Jellyfin access and any other publicly-exposed services.
 
-**Pros:**
+### New path (personal VPN)
-- Simple and fast
-- No extra software
 
-**Cons:**
+A separate WireGuard tunnel for personal remote access with restricted firewall rules:
-- Manual key and endpoint management for every peer
-- Doesn't scale well
-- If behind CGNAT, still needs the VPS as intermediary
 
-### 3. Headscale (Self-Hosted Tailscale)
+```
+Laptop/Phone → VPS (WireGuard peers) → tunnel → extgw01 (firewall) → allowed services
+```
 
-Run a Headscale control server (on the VPS or internally) and install the Tailscale client on homelab hosts and personal devices. Gets the Tailscale mesh networking UX without depending on Tailscale's infrastructure.
+### Access tiers
 
-**Pros:**
+1. **VPN (default)**: Laptop/phone connect to VPS WireGuard endpoint, traffic routed through extgw01 firewall. Only whitelisted services are reachable.
-- Mesh topology - devices communicate directly via NAT traversal (DERP relay as fallback)
+2. **SSH + 2FA (escalated)**: SSH into extgw01 for full network access when needed.
-- Easy to add/remove devices
-- ACL support for granular access control
-- MagicDNS for service discovery
-- Good NixOS support for both headscale server and tailscale client
-- Subnet routing lets you expose the entire 10.69.13.x network or specific hosts without installing tailscale on every host
 
-**Cons:**
+## New Host: extgw01
-- More moving parts than plain WireGuard
-- Headscale is a third-party reimplementation, can lag behind Tailscale features
-- Need to run and maintain the control server
 
-### 4. Tailscale (Managed)
+A NixOS host on the internal network acting as both WireGuard gateway and SSH bastion.
 
-Same as Headscale but using Tailscale's hosted control plane.
+### Responsibilities
 
-**Pros:**
+- **WireGuard tunnel** to the VPS for client traffic
-- Zero infrastructure to manage on the control plane side
+- **Firewall** with allowlist controlling which internal services are reachable through the VPN
-- Polished UX, well-maintained clients
+- **SSH bastion** with 2FA for full network access when needed
-- Free tier covers personal use
+- **DNS**: Clients get split DNS config (laptop via systemd-resolved routing domain, phone uses internal DNS for all queries)
 
-**Cons:**
+### Firewall allowlist (initial)
-- Dependency on Tailscale's service
-- Less aligned with self-hosting preference
-- Coordination metadata goes through their servers (data plane is still peer-to-peer)
 
-### 5. Netbird (Self-Hosted)
+| Service    | Destination                  | Port  |
+|------------|------------------------------|-------|
+| Grafana    | monitoring01.home.2rjus.net  | 3000  |
+| Jellyfin   | jelly01.home.2rjus.net       | 8096  |
+| Sonarr     | sonarr-jail.home.2rjus.net   | 8989  |
+| Radarr     | radarr-jail.home.2rjus.net   | 7878  |
+| NZBget     | nzbget-jail.home.2rjus.net   | 6789  |
 
-Open-source alternative to Tailscale with a self-hostable management server. WireGuard-based, supports ACLs and NAT traversal.
+### SSH 2FA options (to be decided)
 
-**Pros:**
+- **Kanidm**: Already deployed on kanidm01, supports RADIUS/OAuth2 for PAM integration
-- Fully self-hostable
+- **SSH certificates via OpenBao**: Fits existing Vault infrastructure, short-lived certs
-- Web UI for management
+- **TOTP via PAM**: Simplest fallback, Google Authenticator / similar
-- ACL and peer grouping support
 
-**Cons:**
+## VPS Configuration
-- Heavier to self-host (needs multiple components: management server, signal server, TURN relay)
-- Less mature NixOS module support compared to Tailscale/Headscale
 
-### 6. Nebula (by Defined Networking)
+The VPS needs a new WireGuard interface (separate from the existing http-proxy tunnel):
 
-Certificate-based mesh VPN. Each node gets a certificate from a CA you control. No central coordination server needed at runtime.
+- WireGuard endpoint listening on a public UDP port
+- 2 peers: laptop, phone
+- Routes client traffic through tunnel to extgw01
+- Minimal config - just routing, no firewall policy (that lives on extgw01)
 
-**Pros:**
+## Implementation Steps
-- No always-on control plane
-- Certificate-based identity
-- Lightweight
 
-**Cons:**
+1. **Create extgw01 host configuration** in this repo
-- Less convenient for ad-hoc device addition (need to issue certs)
+   - VM provisioned via OpenTofu (same as other hosts)
-- NAT traversal less mature than Tailscale's
+   - WireGuard interface for VPS tunnel
-- Smaller community/ecosystem
+   - nftables/iptables firewall with service allowlist
-
+   - IP forwarding enabled
-## Key Decision Points
+2. **Configure VPS WireGuard** for client peers
-
+   - New WireGuard interface with laptop + phone peers
-- **Static public IP vs CGNAT?** Determines whether clients can connect directly to home network or need VPS relay.
+   - Routing for 10.69.13.0/24 through extgw01 tunnel
-- **Number of client devices?** If just phone and laptop, plain WireGuard via VPS is fine. More devices favors Headscale.
+3. **Set up client configs**
-- **Per-service vs per-network access?** Gateway with firewall rules gives per-service control. Headscale ACLs can also do this. Plain WireGuard gives network-level access with gateway firewall for finer control.
+   - Laptop: WireGuard config + systemd-resolved split DNS for `home.2rjus.net`
-- **Subnet routing vs per-host agents?** With Headscale/Tailscale, can either install client on every host, or use a single subnet router that advertises the 10.69.13.x range. The latter is closer to the gateway approach and avoids touching every host.
+   - Phone: WireGuard app config with DNS pointing at internal nameservers
-
+4. **Set up SSH 2FA** on extgw01
-## Leading Candidates
+   - Evaluate Kanidm integration vs OpenBao SSH certs vs TOTP
-
+5. **Test and verify**
-Based on existing WireGuard experience, self-hosting preference, and NixOS stack:
+   - VPN access to allowed services only
-
+   - Firewall blocks everything else
-1. **Headscale with a subnet router** - Best balance of convenience and self-hosting
+   - SSH + 2FA grants full access
-2. **WireGuard gateway via VPS** - Simplest, most transparent, builds on existing setup
+   - Existing public access path unaffected