# Magicman: PS/2 Keyboard Broken at LUKS Prompt After BIOS Update ## Issue After updating the ThinkPad L14 Gen 4 (21H2S3US00) BIOS to version R24ET51W (1.34) via `fwupdmgr`, the built-in laptop keyboard no longer works during the LUKS disk encryption password prompt. An external USB keyboard must be used to unlock the disk. The laptop keyboard works normally after boot. ## Machine Details - **Model**: Lenovo ThinkPad L14 Gen 4 (21H2S3US00) - **BIOS**: R24ET51W (1.34), dated 2025-10-31 - **EC**: R24HT33W - **Date**: 2026-03-06 ### What fwupdmgr Installed - System Firmware: 0.1.12 → 0.1.34 - UEFI dbx: 20230301 → 20250902 - KEK CA: 2011 → 2023 ## Symptoms - Laptop keyboard does not respond at the LUKS password prompt (neither systemd nor scripted initrd) - USB keyboard works fine at the LUKS prompt - Laptop keyboard works immediately after boot (at greetd login) - Text typed on the laptop keyboard during LUKS prompt sometimes partially appears at the greetd username field after boot, indicating the keyboard hardware IS generating scancodes that get buffered and flushed later ## Kernel Errors Every boot shows these errors from the atkbd driver: ``` atkbd serio0: Failed to deactivate keyboard on isa0060/serio0 atkbd serio0: Failed to enable keyboard on isa0060/serio0 input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 atkbd serio0: Spurious ACK on isa0060/serio0. Some program might be trying to access hardware directly. ``` The keyboard device IS registered despite the errors, and the `kbd` input handler binds to it (`Handlers=sysrq kbd leds event6`). ## Root Cause Analysis The BIOS update changed the PS/2 controller (i8042) initialization behavior. The atkbd driver sends a deactivate command (0xF5) during init, which likely succeeds at disabling the keyboard even though the ACK times out. The subsequent enable command (0xF4) also times out without re-enabling it. The keyboard stays disabled at the hardware level — it queues keypresses in its small internal buffer (~16 keys) but doesn't send scancodes to the host until something re-enables it during full boot. This is NOT a timing issue — leaving the system at the LUKS prompt for several minutes does not fix the keyboard. Something specific that happens later in the boot process (likely during switch-root when udev re-processes devices) re-enables the keyboard. ## What Was Tried ### Kernel Parameters (none helped) - `i8042.dumbkbd` — skip keyboard reset during i8042 probe - `i8042.nopnp` — don't use PNP to discover controllers - `i8042.reset` — force i8042 controller reset - `i8042.nomux` — don't probe for MUX - `atkbd.reset` — reset keyboard during atkbd init - `console=tty1` — explicitly route console I/O to tty1 - Various combinations of the above ### Initrd Module Loading - Added `i8042`, `atkbd`, `thinkpad_acpi` to `boot.initrd.kernelModules` - `thinkpad_acpi` loads the EC driver early, but didn't help ### Initrd Services - Created `keyboard-reconnect` systemd service that runs before `systemd-cryptsetup@root.service` - Tried `echo reconnect > /sys/bus/serio/devices/serio0/drvctl` — reconnect also fails - Tried full module reload: `rmmod atkbd; rmmod i8042; sleep N; modprobe i8042; modprobe atkbd` - Tested with sleep 2 and sleep 8 - The reload creates a new serio device (serio2) but initialization fails identically ### Plymouth - Disabled Plymouth (`boot.plymouth.enable = false`) — no effect - Tested `plymouth.enable=0` on kernel command line — no effect - Confirmed password agent falls back to `systemd-tty-ask-password-agent` on `/dev/tty1` ### Scripted Initrd - Switched from systemd initrd to scripted initrd (`boot.initrd.systemd.enable = false`) - Uses a completely different password prompt mechanism (shell `read`) - Same result — keyboard still doesn't work ### BIOS - Checked BIOS settings — no relevant keyboard/PS/2 options available - `fwupdmgr get-updates` shows no newer BIOS version available ## Planned Fix: TPM + Secure Boot Auto-Unlock ### Approach Use TPM2-based LUKS unlock with Secure Boot to bypass the keyboard requirement entirely. - **lanzaboote** — replaces systemd-boot, produces signed Unified Kernel Images (UKIs) that bundle kernel + initrd + cmdline into a single signed EFI binary - **Secure Boot** — ensures only signed code can boot, prevents tampering with boot chain - **TPM2 unlock** — `systemd-cryptenroll` binds LUKS key to TPM PCR 7 (Secure Boot policy) - **Passphrase kept as fallback** — if TPM/Secure Boot state changes, unlock with USB keyboard + password ### Why PCR 7 Only Binding to PCR 7 alone means kernel/initrd updates (frequent on nixos-unstable) do NOT require re-enrollment. PCR 7 only changes when Secure Boot keys or policy change. Cmdline tampering is prevented by lanzaboote's UKI approach — the cmdline is embedded in the signed binary and cannot be edited at the bootloader. ### Setup Steps 1. Install `sbctl` and create Secure Boot signing keys 2. Put BIOS into Secure Boot "Setup Mode" and enroll custom keys (include Microsoft keys for fwupd) 3. Enable lanzaboote in NixOS config (replaces systemd-boot) 4. Rebuild, verify Secure Boot works 5. Enroll TPM with `systemd-cryptenroll --tpm2-device=auto --tpm2-pcrs=7` After setup, `nixos-rebuild switch/boot` works as usual — lanzaboote automatically signs each new generation. ### Security Considerations **Protected against:** - Offline disk read (pull SSD, boot USB) - Boot chain tampering (unsigned code won't boot) - Cmdline editing (locked into signed UKI) **Remaining attack surface:** - Stolen while suspended — disk is decrypted in RAM, only screen lock protects. Consider hibernate instead of suspend (hibernate locks LUKS since RAM is powered off). - Network services — system is fully running after boot, exposed services are reachable - DMA attacks via Thunderbolt/PCIe — mitigated by IOMMU (should be on by default) - Cold boot attacks — exotic, requires freezing RAM For a stolen-laptop scenario this is solid. The biggest practical risk is theft while the laptop is suspended. ### Other Considered Alternatives - **BIOS update from Lenovo** fixing the PS/2 controller init sequence - **Kernel patch** to handle the failed enable more gracefully - **TPM + PIN** — not viable due to the same PS/2 keyboard issue at the PIN prompt ## Current Workaround Use an external USB keyboard to enter the LUKS password at boot.