Torjus Håkestad
859e35ab5c
Add 3 opt-in Loki tools (query_logs, list_labels, list_label_values) that are registered when LOKI_URL is configured. Includes Loki HTTP client, CLI commands (logs, labels), NixOS module option, formatting, and tests. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
362 lines
16 KiB
Markdown
362 lines
16 KiB
Markdown
# CLAUDE.md - Project Context
|
|
|
|
This file provides context for Claude when working on this project.
|
|
|
|
## Project Overview
|
|
|
|
**LabMCP** is a collection of Model Context Protocol (MCP) servers written in Go, designed to extend Claude's capabilities with custom tools. The repository is structured to be generic and extensible, allowing multiple MCP servers to be added over time.
|
|
|
|
## MCP Servers
|
|
|
|
### Nixpkgs Search (`nixpkgs-search`) - **Primary**
|
|
Combined search for NixOS options and Nix packages from nixpkgs. Provides two separate MCP servers:
|
|
- **Options server**: Search NixOS configuration options (`nixpkgs-search options serve`)
|
|
- **Packages server**: Search Nix packages (`nixpkgs-search packages serve`)
|
|
|
|
### NixOS Options (`nixos-options`) - Legacy
|
|
Search and query NixOS configuration options. Uses nixpkgs as source.
|
|
*Note: Prefer using `nixpkgs-search options` instead.*
|
|
|
|
### Home Manager Options (`hm-options`)
|
|
Search and query Home Manager configuration options. Uses home-manager repository as source.
|
|
|
|
### Lab Monitoring (`lab-monitoring`)
|
|
Query Prometheus metrics, Alertmanager alerts, and Loki logs. Unlike other servers, this queries live HTTP APIs — no database or indexing needed.
|
|
- 8 core tools: list/get alerts, search metrics, get metadata, PromQL query, list targets, list/create silences
|
|
- 3 optional Loki tools (when `LOKI_URL` is set): query_logs, list_labels, list_label_values
|
|
- Configurable Prometheus, Alertmanager, and Loki URLs via flags or environment variables
|
|
|
|
The nixpkgs/options/hm servers share a database-backed architecture:
|
|
- Full-text search across option/package names and descriptions
|
|
- Query specific options/packages with full metadata
|
|
- Index multiple revisions (by git hash or channel name)
|
|
- Fetch module source files
|
|
- PostgreSQL and SQLite backends
|
|
|
|
## Technology Stack
|
|
|
|
- **Language**: Go 1.24+
|
|
- **Build System**: Nix flakes
|
|
- **Databases**: PostgreSQL and SQLite (both fully supported)
|
|
- **Protocol**: MCP (Model Context Protocol) - JSON-RPC over STDIO or HTTP/SSE
|
|
- **Module Path**: `git.t-juice.club/torjus/labmcp`
|
|
|
|
## Project Status
|
|
|
|
**Complete and maintained** - All core features implemented:
|
|
- Full MCP servers (6 tools each for nixpkgs/options, 8-11 tools for monitoring)
|
|
- PostgreSQL and SQLite backends with FTS (for nixpkgs/options servers)
|
|
- Live API queries for Prometheus/Alertmanager/Loki (monitoring server)
|
|
- NixOS modules for deployment
|
|
- CLI for manual operations
|
|
- Comprehensive test suite
|
|
|
|
## Repository Structure
|
|
|
|
```
|
|
labmcp/
|
|
├── cmd/
|
|
│ ├── nixpkgs-search/
|
|
│ │ └── main.go # Combined options+packages CLI (primary)
|
|
│ ├── nixos-options/
|
|
│ │ └── main.go # NixOS options CLI (legacy)
|
|
│ ├── hm-options/
|
|
│ │ └── main.go # Home Manager options CLI
|
|
│ └── lab-monitoring/
|
|
│ └── main.go # Prometheus/Alertmanager CLI
|
|
├── internal/
|
|
│ ├── database/
|
|
│ │ ├── interface.go # Store interface (options + packages)
|
|
│ │ ├── schema.go # Schema versioning
|
|
│ │ ├── postgres.go # PostgreSQL implementation
|
|
│ │ ├── sqlite.go # SQLite implementation
|
|
│ │ └── *_test.go # Database tests
|
|
│ ├── mcp/
|
|
│ │ ├── server.go # MCP server core + ServerConfig + modes
|
|
│ │ ├── handlers.go # Tool implementations (options + packages)
|
|
│ │ ├── types.go # Protocol types
|
|
│ │ ├── transport.go # Transport interface
|
|
│ │ ├── transport_stdio.go # STDIO transport
|
|
│ │ ├── transport_http.go # HTTP/SSE transport
|
|
│ │ ├── session.go # HTTP session management
|
|
│ │ └── *_test.go # MCP tests
|
|
│ ├── options/
|
|
│ │ └── indexer.go # Shared Indexer interface
|
|
│ ├── nixos/
|
|
│ │ ├── indexer.go # NixOS options indexing
|
|
│ │ ├── parser.go # options.json parsing
|
|
│ │ ├── types.go # Channel aliases, extensions
|
|
│ │ └── *_test.go # Indexer tests
|
|
│ ├── homemanager/
|
|
│ │ ├── indexer.go # Home Manager indexing
|
|
│ │ ├── types.go # Channel aliases, extensions
|
|
│ │ └── *_test.go # Indexer tests
|
|
│ ├── packages/
|
|
│ │ ├── indexer.go # Nix packages indexing
|
|
│ │ ├── parser.go # nix-env JSON parsing
|
|
│ │ ├── types.go # Package types, channel aliases
|
|
│ │ └── *_test.go # Parser tests
|
|
│ └── monitoring/
|
|
│ ├── types.go # Prometheus/Alertmanager/Loki API types
|
|
│ ├── prometheus.go # Prometheus HTTP client
|
|
│ ├── alertmanager.go # Alertmanager HTTP client
|
|
│ ├── loki.go # Loki HTTP client
|
|
│ ├── handlers.go # MCP tool definitions + handlers
|
|
│ ├── format.go # Markdown formatting utilities
|
|
│ └── *_test.go # Tests (httptest-based)
|
|
├── nix/
|
|
│ ├── module.nix # NixOS module for nixos-options
|
|
│ ├── hm-options-module.nix # NixOS module for hm-options
|
|
│ ├── lab-monitoring-module.nix # NixOS module for lab-monitoring
|
|
│ └── package.nix # Parameterized Nix package
|
|
├── testdata/
|
|
│ └── options-sample.json # Test fixture
|
|
├── flake.nix
|
|
├── go.mod
|
|
├── .mcp.json # MCP client configuration
|
|
├── CLAUDE.md # This file
|
|
├── README.md
|
|
└── TODO.md # Future improvements
|
|
```
|
|
|
|
## MCP Tools
|
|
|
|
### Options Servers (nixpkgs-search options, nixos-options, hm-options)
|
|
|
|
| Tool | Description |
|
|
|------|-------------|
|
|
| `search_options` | Full-text search across option names and descriptions |
|
|
| `get_option` | Get full details for a specific option with children |
|
|
| `get_file` | Fetch source file contents from indexed repository |
|
|
| `index_revision` | Index a revision (by hash or channel name) |
|
|
| `list_revisions` | List all indexed revisions |
|
|
| `delete_revision` | Delete an indexed revision |
|
|
|
|
### Packages Server (nixpkgs-search packages)
|
|
|
|
| Tool | Description |
|
|
|------|-------------|
|
|
| `search_packages` | Full-text search across package names and descriptions |
|
|
| `get_package` | Get full details for a specific package by attr path |
|
|
| `get_file` | Fetch source file contents from nixpkgs |
|
|
| `list_revisions` | List all indexed revisions |
|
|
| `delete_revision` | Delete an indexed revision |
|
|
|
|
### Monitoring Server (lab-monitoring)
|
|
|
|
| Tool | Description |
|
|
|------|-------------|
|
|
| `list_alerts` | List alerts with optional filters (state, severity, receiver) |
|
|
| `get_alert` | Get full details for a specific alert by fingerprint |
|
|
| `search_metrics` | Search metric names with substring filter, enriched with metadata |
|
|
| `get_metric_metadata` | Get type, help text, and unit for a specific metric |
|
|
| `query` | Execute instant PromQL query |
|
|
| `list_targets` | List scrape targets with health status |
|
|
| `list_silences` | List active/pending silences |
|
|
| `create_silence` | Create a silence (confirms with user first) |
|
|
| `query_logs` | Execute a LogQL range query against Loki (requires `LOKI_URL`) |
|
|
| `list_labels` | List available label names from Loki (requires `LOKI_URL`) |
|
|
| `list_label_values` | List values for a specific label from Loki (requires `LOKI_URL`) |
|
|
|
|
## Key Implementation Details
|
|
|
|
### Database
|
|
- Schema versioning with automatic recreation on version mismatch
|
|
- Full-text search: SQLite FTS5, PostgreSQL tsvector/GIN
|
|
- Path-based queries use LIKE for exact prefix matching
|
|
- Batch operations for efficient indexing
|
|
|
|
### Indexing
|
|
- Uses `nix-build` to evaluate options from any revision
|
|
- File indexing downloads tarball and stores allowed extensions (.nix, .json, .md, etc.)
|
|
- File indexing enabled by default (use `--no-files` to skip)
|
|
- Skips already-indexed revisions (use `--force` to re-index)
|
|
|
|
### Transports
|
|
- **STDIO**: Default transport, line-delimited JSON-RPC (for CLI/desktop MCP clients)
|
|
- **HTTP**: Streamable HTTP transport with SSE (for web-based MCP clients)
|
|
- Session management with cryptographically secure IDs
|
|
- Configurable CORS (localhost-only by default)
|
|
- Optional TLS support
|
|
- SSE keepalive messages (15s default)
|
|
|
|
### Security
|
|
- Revision parameter validated against strict regex to prevent Nix injection
|
|
- Path traversal protection using `filepath.Clean()` and `filepath.IsAbs()`
|
|
- NixOS module supports `connectionStringFile` for PostgreSQL secrets
|
|
- Systemd service runs with extensive hardening options
|
|
- HTTP transport hardening:
|
|
- Request body size limit (1MB default)
|
|
- Server timeouts (read: 30s, write: 30s, idle: 120s, header: 10s)
|
|
- Maximum session limit (10,000 default)
|
|
- Origin validation for CORS
|
|
|
|
## CLI Commands
|
|
|
|
### nixpkgs-search (Primary)
|
|
```bash
|
|
# Options MCP Server
|
|
nixpkgs-search options serve # Run options MCP server on STDIO
|
|
nixpkgs-search options search <query> # Search options
|
|
nixpkgs-search options get <option> # Get option details
|
|
|
|
# Packages MCP Server
|
|
nixpkgs-search packages serve # Run packages MCP server on STDIO
|
|
nixpkgs-search packages search <query> # Search packages
|
|
nixpkgs-search packages get <attr> # Get package details
|
|
|
|
# Combined Indexing
|
|
nixpkgs-search index <revision> # Index options AND packages
|
|
nixpkgs-search index --no-packages <r> # Index options only (faster)
|
|
nixpkgs-search index --no-options <r> # Index packages only
|
|
nixpkgs-search index --no-files <r> # Skip file indexing
|
|
nixpkgs-search index --force <r> # Force re-index
|
|
|
|
# Shared Commands
|
|
nixpkgs-search list # List indexed revisions
|
|
nixpkgs-search delete <revision> # Delete indexed revision
|
|
nixpkgs-search --version # Show version
|
|
```
|
|
|
|
### nixos-options (Legacy)
|
|
```bash
|
|
nixos-options serve # Run MCP server on STDIO (default)
|
|
nixos-options serve --transport http # Run MCP server on HTTP
|
|
nixos-options index <revision> # Index a nixpkgs revision
|
|
nixos-options index --force <r> # Force re-index existing revision
|
|
nixos-options index --no-files # Skip file content indexing
|
|
nixos-options list # List indexed revisions
|
|
nixos-options search <query> # Search options
|
|
nixos-options get <option> # Get option details
|
|
nixos-options delete <revision> # Delete indexed revision
|
|
nixos-options --version # Show version
|
|
```
|
|
|
|
### hm-options
|
|
```bash
|
|
hm-options serve # Run MCP server on STDIO (default)
|
|
hm-options serve --transport http # Run MCP server on HTTP
|
|
hm-options index <revision> # Index a home-manager revision
|
|
hm-options index --force <r> # Force re-index existing revision
|
|
hm-options index --no-files # Skip file content indexing
|
|
hm-options list # List indexed revisions
|
|
hm-options search <query> # Search options
|
|
hm-options get <option> # Get option details
|
|
hm-options delete <revision> # Delete indexed revision
|
|
hm-options --version # Show version
|
|
```
|
|
|
|
### lab-monitoring
|
|
```bash
|
|
lab-monitoring serve # Run MCP server on STDIO
|
|
lab-monitoring serve --transport http # Run MCP server on HTTP
|
|
lab-monitoring alerts # List alerts
|
|
lab-monitoring alerts --state active # Filter by state
|
|
lab-monitoring query 'up' # Instant PromQL query
|
|
lab-monitoring targets # List scrape targets
|
|
lab-monitoring metrics node # Search metric names
|
|
lab-monitoring logs '{job="varlogs"}' # Query logs (requires LOKI_URL)
|
|
lab-monitoring logs '{job="nginx"} |= "error"' --start 2h --limit 50
|
|
lab-monitoring labels # List Loki labels
|
|
lab-monitoring labels --values job # List values for a label
|
|
```
|
|
|
|
### Channel Aliases
|
|
|
|
**nixpkgs-search/nixos-options**: `nixos-unstable`, `nixos-stable`, `nixos-24.11`, `nixos-24.05`, etc.
|
|
|
|
**hm-options**: `hm-unstable`, `hm-stable`, `master`, `release-24.11`, `release-24.05`, etc.
|
|
|
|
## Notes for Claude
|
|
|
|
### Planning
|
|
When creating implementation plans, the first step should usually be to **checkout an appropriately named feature branch** (e.g., `git checkout -b feature/lab-monitoring`). This keeps work isolated and makes PRs cleaner.
|
|
|
|
**After implementing a plan**, update the README.md to reflect any new or changed functionality (new servers, tools, CLI commands, configuration options, NixOS module options, etc.).
|
|
|
|
### Development Workflow
|
|
- **Always run `go fmt ./...` before committing Go code**
|
|
- **Run Go commands using `nix develop -c`** (e.g., `nix develop -c go test ./...`)
|
|
- **Use `nix run` to run/test binaries** (e.g., `nix run .#nixpkgs-search -- options serve`)
|
|
- Do NOT use `go build -o /tmp/...` to test binaries - always use `nix run`
|
|
- Remember: modified files must be tracked by git for `nix run` to see them
|
|
- File paths in responses should use format `path/to/file.go:123`
|
|
|
|
### Linting
|
|
**Before completing work on a feature**, run all linting tools to ensure code quality:
|
|
|
|
```bash
|
|
# Run all linters (should report 0 issues)
|
|
nix develop -c golangci-lint run ./...
|
|
|
|
# Check for known vulnerabilities in dependencies
|
|
nix develop -c govulncheck ./...
|
|
|
|
# Run go vet for additional static analysis
|
|
nix develop -c go vet ./...
|
|
```
|
|
|
|
All three tools should pass with no issues before merging a feature branch.
|
|
|
|
### Nix Build Requirement
|
|
**IMPORTANT**: When running `nix build`, `nix run`, or similar commands, new files must be tracked by git first. Nix flakes only see git-tracked files. If you create new files, run `git add <file>` before attempting nix operations.
|
|
|
|
### Version Bumping
|
|
Version bumps should be done once per feature branch, not per commit. Rules:
|
|
- **Patch bump** (0.1.0 → 0.1.1): Changes to Go code within `internal/` that affect a program
|
|
- **Minor bump** (0.1.0 → 0.2.0): Changes to Go code outside `internal/` (e.g., `cmd/`)
|
|
- **Major bump** (0.1.0 → 1.0.0): Breaking changes to CLI usage or MCP protocol
|
|
|
|
Version is defined in multiple places that must stay in sync:
|
|
- `cmd/nixpkgs-search/main.go`
|
|
- `cmd/nixos-options/main.go`
|
|
- `cmd/hm-options/main.go`
|
|
- `cmd/lab-monitoring/main.go`
|
|
- `internal/mcp/server.go` (in `DefaultNixOSConfig`, `DefaultHomeManagerConfig`, `DefaultNixpkgsPackagesConfig`, `DefaultMonitoringConfig`)
|
|
- `nix/package.nix`
|
|
|
|
### User Preferences
|
|
- User prefers PostgreSQL over SQLite (has homelab infrastructure)
|
|
- User values good test coverage and benchmarking
|
|
- Project should remain generic to support future MCP servers
|
|
|
|
### Testing
|
|
```bash
|
|
# Run all tests
|
|
nix develop -c go test ./... -short
|
|
|
|
# Run with verbose output
|
|
nix develop -c go test ./... -v
|
|
|
|
# Run benchmarks (requires nix-build)
|
|
nix develop -c go test -bench=. -benchtime=1x -timeout=30m ./internal/nixos/...
|
|
nix develop -c go test -bench=. -benchtime=1x -timeout=30m ./internal/homemanager/...
|
|
```
|
|
|
|
### Building
|
|
```bash
|
|
# Build with nix
|
|
nix build .#nixpkgs-search
|
|
nix build .#nixos-options
|
|
nix build .#hm-options
|
|
nix build .#lab-monitoring
|
|
|
|
# Run directly
|
|
nix run .#nixpkgs-search -- options serve
|
|
nix run .#nixpkgs-search -- packages serve
|
|
nix run .#nixpkgs-search -- index nixos-unstable
|
|
nix run .#hm-options -- serve
|
|
nix run .#hm-options -- index hm-unstable
|
|
nix run .#lab-monitoring -- serve
|
|
```
|
|
|
|
### Indexing Performance
|
|
Indexing operations are slow due to Nix evaluation and file downloads. When running index commands, use appropriate timeouts:
|
|
- **nixpkgs-search (full)**: ~15-20 minutes for `nixos-unstable` (options + packages + files)
|
|
- **nixpkgs-search (options only)**: ~5-6 minutes with `--no-packages`
|
|
- **nixpkgs-search (packages only)**: ~10-15 minutes with `--no-options`
|
|
- **hm-options**: ~1-2 minutes for `master` (with files)
|
|
|
|
Use `--no-files` flag to skip file indexing for faster results.
|
|
Use `--no-packages` to index only options (matches legacy behavior).
|