system: add pipe-to-loki helper script

Adds a system-wide script for sending command output or interactive
sessions to Loki for easy sharing with Claude.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

docs/plans/auth-system-replacement.md

@@ -66,9 +66,9 @@ This future migration path is a strong argument for Kanidm over LDAP-only soluti
    - Vault integration for idm_admin password
    - LDAPS on port 636
 
-2. **Configure declarative provisioning** ✅
-   - Groups: `admins`, `users`, `ssh-users`
-   - User: `torjus` (member of all groups)
+2. **Configure provisioning** ✅
+   - Groups provisioned declaratively: `admins`, `users`, `ssh-users`
+   - Users managed imperatively via CLI (allows setting POSIX passwords in one step)
    - POSIX attributes enabled (UID/GID range 65,536-69,999)
 
 3. **Test NAS integration** (in progress)
@@ -80,14 +80,16 @@ This future migration path is a strong argument for Kanidm over LDAP-only soluti
    - Grafana
    - Other services as needed
 
-5. **Create client module** in `system/` for PAM/NSS
-   - Enable on all hosts that need central auth
-   - Configure trusted CA
+5. **Create client module** in `system/` for PAM/NSS ✅
+   - Module: `system/kanidm-client.nix`
+   - `homelab.kanidm.enable = true` enables PAM/NSS
+   - Short usernames (not SPN format)
+   - Home directory symlinks via `home_alias`
+   - Enabled on test tier: testvm01, testvm02, testvm03
 
-6. **Documentation**
-   - User management procedures
-   - Adding new OAuth2 clients
-   - Troubleshooting PAM/NSS issues
+6. **Documentation** ✅
+   - `docs/user-management.md` - CLI workflows, troubleshooting
+   - User/group creation procedures verified working
 
 ## Progress
 
@@ -106,14 +108,37 @@ This future migration path is a strong argument for Kanidm over LDAP-only soluti
 - Prometheus monitoring scrape target configured
 
 **Provisioned entities:**
-- Groups: `admins`, `users`, `ssh-users`
-- User: `torjus` (member of all groups, POSIX enabled with GID 65536)
+- Groups: `admins`, `users`, `ssh-users` (declarative)
+- Users managed via CLI (imperative)
 
 **Verified working:**
 - WebUI login with idm_admin
 - LDAP bind and search with POSIX-enabled user
 - LDAPS with valid internal CA certificate
 
+### Completed (2026-02-08) - PAM/NSS Client
+
+**Client module deployed (`system/kanidm-client.nix`):**
+- `homelab.kanidm.enable = true` enables PAM/NSS integration
+- Connects to auth.home.2rjus.net
+- Short usernames (`torjus` instead of `torjus@home.2rjus.net`)
+- Home directory symlinks (`/home/torjus` → UUID-based dir)
+- Login restricted to `ssh-users` group
+
+**Enabled on test tier:**
+- testvm01, testvm02, testvm03
+
+**Verified working:**
+- User/group resolution via `getent`
+- SSH login with Kanidm unix passwords
+- Home directory creation with symlinks
+- Imperative user/group creation via CLI
+
+**Documentation:**
+- `docs/user-management.md` with full CLI workflows
+- Password requirements (min 10 chars)
+- Troubleshooting guide (nscd, cache invalidation)
+
 ### UID/GID Range (Resolved)
 
 **Range: 65,536 - 69,999** (manually allocated)
@@ -128,10 +153,9 @@ Rationale:
 
 ### Next Steps
 
-1. Deploy to monitoring01 to enable Prometheus scraping
+1. Enable PAM/NSS on production hosts (after test tier validation)
 2. Configure TrueNAS LDAP client for NAS integration testing
 3. Add OAuth2 clients (Grafana first)
-4. Create PAM/NSS client module for other hosts
 
 ## References
 

docs/user-management.md

@@ -21,61 +21,113 @@ kanidm login --name idm_admin --url https://auth.home.2rjus.net
 
 ## User Management
 
-### Creating Users
+POSIX users are managed imperatively via the `kanidm` CLI. This allows setting
+all attributes (including UNIX password) in one workflow.
 
-Users are provisioned declaratively in `services/kanidm/default.nix`:
-
-```nix
-services.kanidm.provision.persons.username = {
-  displayName = "Display Name";
-  groups = [ "admins" "users" "ssh-users" ];
-};
-```
-
-### Enabling POSIX for Users
-
-For PAM/NSS integration, users need POSIX attributes and a UNIX password:
+### Creating a POSIX User
 
 ```bash
-# Check if user has POSIX enabled
-kanidm person get <username>
+# Create the person
+kanidm person create <username> "<Display Name>"
 
-# Set UNIX password (required for SSH login)
+# Add to groups
+kanidm group add-members ssh-users <username>
+
+# Enable POSIX (UID is auto-assigned)
+kanidm person posix set <username>
+
+# Set UNIX password (required for SSH login, min 10 characters)
 kanidm person posix set-password <username>
+
+# Optionally set login shell
+kanidm person posix set <username> --shell /bin/zsh
+```
+
+### Example: Full User Creation
+
+```bash
+kanidm person create testuser "Test User"
+kanidm group add-members ssh-users testuser
+kanidm person posix set testuser
+kanidm person posix set-password testuser
+kanidm person get testuser
+```
+
+After creation, verify on a client host:
+```bash
+getent passwd testuser
+ssh testuser@testvm01.home.2rjus.net
+```
+
+### Viewing User Details
+
+```bash
+kanidm person get <username>
+```
+
+### Removing a User
+
+```bash
+kanidm person delete <username>
 ```
 
 ## Group Management
 
-### Creating Groups
+Groups for POSIX access are also managed via CLI.
 
-Groups are provisioned declaratively:
-
-```nix
-services.kanidm.provision.groups = {
-  admins = { };
-  users = { };
-  ssh-users = { };
-};
-```
-
-### Enabling POSIX for Groups
-
-Groups must have POSIX enabled to be resolved via NSS:
+### Creating a POSIX Group
 
 ```bash
-# Enable POSIX on a group with a specific GID
-kanidm group posix set <group-name> --gidnumber <gid>
+# Create the group
+kanidm group create <group-name>
 
-# Example: enable ssh-users group
-kanidm group posix set ssh-users --gidnumber 68000
+# Enable POSIX with a specific GID
+kanidm group posix set <group-name> --gidnumber <gid>
 ```
 
+### Adding Members
+
+```bash
+kanidm group add-members <group-name> <username>
+```
+
+### Viewing Group Details
+
+```bash
+kanidm group get <group-name>
+kanidm group list-members <group-name>
+```
+
+### Example: Full Group Creation
+
+```bash
+kanidm group create testgroup
+kanidm group posix set testgroup --gidnumber 68010
+kanidm group add-members testgroup testuser
+kanidm group get testgroup
+```
+
+After creation, verify on a client host:
+```bash
+getent group testgroup
+```
+
+### Current Groups
+
+| Group | GID | Purpose |
+|-------|-----|---------|
+| ssh-users | 68000 | SSH login access |
+| admins | 68001 | Administrative access |
+| users | 68002 | General users |
+
 ### UID/GID Allocation
 
+Kanidm auto-assigns UIDs/GIDs from its configured range. For manually assigned GIDs:
+
 | Range | Purpose |
 |-------|---------|
-| 65,536 - 67,999 | Users |
-| 68,000 - 69,999 | Groups |
+| 65,536+ | Users (auto-assigned) |
+| 68,000 - 68,999 | Groups (manually assigned) |
 
 ## PAM/NSS Client Configuration
 
@@ -89,6 +141,12 @@ This configures:
 - `services.kanidm.enablePam = true`
 - Client connection to auth.home.2rjus.net
 - Login authorization for `ssh-users` group
+- Short usernames (`torjus` instead of `torjus@home.2rjus.net`)
+- Home directory symlinks (`/home/torjus` → UUID-based directory)
+
+### Enabled Hosts
+
+- testvm01, testvm02, testvm03 (test tier)
 
 ### Options
 
@@ -100,6 +158,17 @@ homelab.kanidm = {
 };
 ```
 
+### Home Directories
+
+Home directories use UUID-based paths for stability (so renaming a user doesn't
+require moving their home directory). Symlinks provide convenient access:
+
+```
+/home/torjus -> /home/e4f4c56c-4aee-4c20-846f-90cb69807733
+```
+
+The symlinks are created by `kanidm-unixd-tasks` on first login.
+
 ## Testing
 
 ### Verify NSS Resolution
@@ -153,12 +222,46 @@ kanidm group posix set ssh-users --gidnumber 68000
    systemctl status kanidm-unixd
    ```
 
-2. Check client can reach server:
+2. Check unixd can reach server:
+   ```bash
+   kanidm-unix status
+   # Should show: system: online, Kanidm: online
+   ```
+
+3. Check client can reach server:
    ```bash
    curl -s https://auth.home.2rjus.net/status
    ```
 
-3. Check user has POSIX enabled on server:
+4. Check user has POSIX enabled on server:
    ```bash
    kanidm person get <username>
    ```
+
+5. Restart nscd to clear stale cache:
+   ```bash
+   systemctl restart nscd
+   ```
+
+6. Invalidate kanidm cache:
+   ```bash
+   kanidm-unix cache-invalidate
+   ```
+
+### Changes not taking effect after deployment
+
+NixOS uses nsncd (a Rust reimplementation of nscd) for NSS caching. After deploying
+kanidm-unixd config changes, you may need to restart both services:
+
+```bash
+systemctl restart kanidm-unixd
+systemctl restart nscd
+```
+
+### Test PAM authentication directly
+
+Use the kanidm-unix CLI to test PAM auth without SSH:
+
+```bash
+kanidm-unix auth-test --name <username>
+```

services/kanidm/default.nix

@@ -17,7 +17,8 @@
       };
     };
 
-    # Provisioning - initial users/groups
+    # Provision base groups only - users are managed via CLI
+    # See docs/user-management.md for details
     provision = {
       enable = true;
       idmAdminPasswordFile = config.vault.secrets.kanidm-idm-admin.outputDir;
@@ -28,10 +29,7 @@
         ssh-users = { };
       };
 
-      persons.torjus = {
-        displayName = "Torjus";
-        groups = [ "admins" "users" "ssh-users" ];
-      };
+      # Regular users (persons) are managed imperatively via kanidm CLI
     };
   };
 
@@ -46,7 +44,7 @@
     extraDomainNames = [ "${config.networking.hostName}.home.2rjus.net" ];
   };
 
-  # Vault secret for idm_admin password
+  # Vault secret for idm_admin password (used for provisioning)
   vault.secrets.kanidm-idm-admin = {
     secretPath = "kanidm/idm-admin-password";
     extractKey = "password";

system/default.nix

@@ -9,6 +9,7 @@
     ./motd.nix
     ./packages.nix
     ./nix.nix
+    ./pipe-to-loki.nix
     ./root-user.nix
     ./pki/root-ca.nix
     ./sshd.nix

system/pipe-to-loki.nix

@@ -0,0 +1,140 @@
+{
+  config,
+  pkgs,
+  lib,
+  ...
+}:
+let
+  pipe-to-loki = pkgs.writeShellApplication {
+    name = "pipe-to-loki";
+    runtimeInputs = with pkgs; [
+      curl
+      jq
+      util-linux
+      coreutils
+    ];
+    text = ''
+      set -euo pipefail
+
+      LOKI_URL="http://monitoring01.home.2rjus.net:3100/loki/api/v1/push"
+      HOSTNAME=$(hostname)
+      SESSION_ID=""
+      RECORD_MODE=false
+
+      usage() {
+        echo "Usage: pipe-to-loki [--id ID] [--record]"
+        echo ""
+        echo "Send command output or interactive sessions to Loki."
+        echo ""
+        echo "Options:"
+        echo "  --id ID      Set custom session ID (default: auto-generated)"
+        echo "  --record     Start interactive recording session"
+        echo ""
+        echo "Examples:"
+        echo "  command | pipe-to-loki           # Pipe command output"
+        echo "  command | pipe-to-loki --id foo  # Pipe with custom ID"
+        echo "  pipe-to-loki --record            # Start recording session"
+        exit 1
+      }
+
+      generate_id() {
+        local random_chars
+        random_chars=$(head -c 2 /dev/urandom | od -An -tx1 | tr -d ' \n')
+        echo "''${HOSTNAME}-$(date +%s)-''${random_chars}"
+      }
+
+      send_to_loki() {
+        local content="$1"
+        local type="$2"
+        local timestamp_ns
+        timestamp_ns=$(date +%s%N)
+
+        local payload
+        payload=$(jq -n \
+          --arg job "pipe-to-loki" \
+          --arg host "$HOSTNAME" \
+          --arg type "$type" \
+          --arg id "$SESSION_ID" \
+          --arg ts "$timestamp_ns" \
+          --arg content "$content" \
+          '{
+            streams: [{
+              stream: {
+                job: $job,
+                host: $host,
+                type: $type,
+                id: $id
+              },
+              values: [[$ts, $content]]
+            }]
+          }')
+
+        if curl -s -X POST "$LOKI_URL" \
+          -H "Content-Type: application/json" \
+          -d "$payload" > /dev/null; then
+          return 0
+        else
+          echo "Error: Failed to send to Loki" >&2
+          return 1
+        fi
+      }
+
+      # Parse arguments
+      while [[ $# -gt 0 ]]; do
+        case $1 in
+          --id)
+            SESSION_ID="$2"
+            shift 2
+            ;;
+          --record)
+            RECORD_MODE=true
+            shift
+            ;;
+          --help|-h)
+            usage
+            ;;
+          *)
+            echo "Unknown option: $1" >&2
+            usage
+            ;;
+        esac
+      done
+
+      # Generate ID if not provided
+      if [[ -z "$SESSION_ID" ]]; then
+        SESSION_ID=$(generate_id)
+      fi
+
+      if $RECORD_MODE; then
+        # Session recording mode
+        SCRIPT_FILE=$(mktemp)
+        trap 'rm -f "$SCRIPT_FILE"' EXIT
+
+        echo "Recording session $SESSION_ID... (exit to send)"
+
+        # Use script to record the session
+        script -q "$SCRIPT_FILE"
+
+        # Read the transcript and send to Loki
+        content=$(cat "$SCRIPT_FILE")
+        if send_to_loki "$content" "session"; then
+          echo "Session $SESSION_ID sent to Loki"
+        fi
+      else
+        # Pipe mode - read from stdin
+        if [[ -t 0 ]]; then
+          echo "Error: No input provided. Pipe a command or use --record for interactive mode." >&2
+          exit 1
+        fi
+
+        content=$(cat)
+        if send_to_loki "$content" "command"; then
+          echo "Sent to Loki with id: $SESSION_ID"
+        fi
+      fi
+    '';
+  };
+in
+{
+  environment.systemPackages = [ pipe-to-loki ];
+}