From 11c300c4e7327995a31c840a05981a8fde29dbac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Torjus=20H=C3=A5kestad?= Date: Tue, 3 Feb 2026 18:53:55 +0100 Subject: [PATCH] docs: update TODO.md with future improvements Replace completed planning document with actionable improvement ideas. Co-Authored-By: Claude Opus 4.5 --- TODO.md | 214 ++++---------------------------------------------------- 1 file changed, 15 insertions(+), 199 deletions(-) diff --git a/TODO.md b/TODO.md index 8d0e379..c4e6686 100644 --- a/TODO.md +++ b/TODO.md @@ -1,207 +1,23 @@ -# LabMCP - MCP Server Collection +# TODO - Future Improvements -## Project Summary +## Quick Wins -This repository hosts a collection of Model Context Protocol (MCP) servers designed to extend Claude's capabilities with custom tools. The project is structured to be generic and extensible, allowing multiple MCP servers to be added over time. +- [ ] Add `--version` flag to the CLI +- [ ] Check if revision exists before indexing (skip or require `--force`) -**Technology Stack:** -- Language: Go -- Build System: Nix/NixOS -- Deployment: Nix flake providing importable packages +## Usability -## Initial MCP Server: NixOS Options Search +- [ ] Progress reporting during indexing ("Fetching nixpkgs... Parsing options... Indexing files...") +- [ ] Add `search_files` MCP tool - search for files by path pattern (e.g., find all nginx-related modules) -The first MCP server will provide search capabilities for NixOS options. This addresses the challenge of incomplete or hard-to-find documentation in the Nix ecosystem by allowing Claude to query available NixOS configuration options directly. +## Robustness -## TODO +- [ ] PostgreSQL integration tests with testcontainers (currently skipped without manual DB setup) +- [ ] Graceful handling of concurrent indexing (what happens if two clients index the same revision?) -### Planning & Architecture -- [ ] Decide on NixOS options data source - - Options: `nixos-option`, JSON exports, search.nixos.org API, etc. -- [ ] Define MCP server protocol implementation approach - - How to structure the Go MCP server - - What tools/resources to expose via MCP -- [ ] Plan repository structure - - Monorepo with multiple servers? - - Shared libraries/utilities? - - How to organize flake outputs +## Nice to Have -### NixOS Options MCP Server -- [ ] Implement basic MCP server in Go -- [ ] Add NixOS options search functionality -- [ ] Define search capabilities (by name, description, type, etc.) -- [ ] Handle option metadata (type, default, example, description) - -### Nix/Build Configuration -- [ ] Set up flake.nix with proper structure -- [ ] Create buildable package for the MCP server -- [ ] Ensure package is importable from other repositories -- [ ] Add development shell for local testing - -### Testing & Documentation -- [ ] Unit tests for all major components - - Database operations (both PostgreSQL and SQLite) - - MCP protocol handling - - NixOS options parsing - - Search functionality - - File retrieval -- [ ] Integration tests - - End-to-end indexing workflow - - MCP tool invocations - - Multi-revision scenarios -- [ ] Benchmarks - - Indexing performance (time to index a full nixpkgs revision) - - Search query performance - - Database query optimization - - Memory usage during indexing -- [ ] Test fixtures - - Sample options.json files - - Mock nixpkgs repositories for testing - - Test databases with known data -- [ ] Test MCP server with Claude (real-world usage) -- [ ] Document usage and installation -- [ ] Add examples of useful queries - -## Decisions Made - -1. **Data Source**: Use NixOS options.json exports, pre-processed and indexed -2. **Database**: Support both PostgreSQL (primary) and SQLite (lightweight option) -3. **Language**: Go with database/sql abstraction layer -4. **Build**: Nix flake with importable packages - -## MCP Tools / Features - -### Core Search & Query -1. **`search_options`** - Fuzzy/partial matching search - - Search by name (e.g., "nginx" matches "services.nginx.*") - - Full-text search in descriptions - - Requires: revision (git hash or alias), query string - - Optional filters: - - Option type (boolean, string, path, package, etc.) - - Namespace/category (services.*, programs.*, etc.) - - Has default value (true/false) - -2. **`get_option`** - Get full details for specific option - - Returns: name, type, default, example, description, declarations (file paths) - - Limit depth to prevent returning thousands of sub-options - - Default: direct children only (one level deep) - - Optional: `depth` parameter for recursive queries - - Show related/nearby options in same namespace - - Requires: revision, option path - -3. **`get_file`** - Fetch nixpkgs source file contents - - Example: `modules/services/web-servers/caddy/default.nix` - - Useful when option descriptions are unclear - - Security: Path validation, no traversal, nixpkgs-only - - Requires: revision, file path - -### Revision Management -4. **`index_revision`** - Index a specific nixpkgs revision - - Takes: git hash (full or short) - - Fetches nixpkgs, extracts options.json, populates database - - Stores metadata: hash, date, channel name (if known) - - May be slow - consider async operation - -5. **`list_revisions`** - List indexed revisions - - Returns: git hash, date, channel name, option count - - Helps users see what's available before querying - -6. **`delete_revision`** - Prune old/unused revisions - - Remove revision and all associated data from database - - Useful for DB maintenance and disk space management - -### Channel/Tag Support -- Support friendly aliases: `nixos-unstable`, `nixos-24.05`, `nixos-23.11` -- Can auto-update these periodically or on-demand -- Allow searching by channel name instead of git hash - -## Database Schema - -**Tables:** -- `revisions` - nixpkgs git hash, date, channel name, metadata -- `options` - per revision: name, type, default, example, description -- `declarations` - file paths where options are declared -- `files` (optional) - cache of nixpkgs file contents for faster access - -**Indexes:** -- Full-text search on option names and descriptions -- B-tree indexes on revision + option name for fast lookups -- Namespace prefix indexes for category filtering - -## Implementation Considerations - -**Result Limiting:** -- When getting options, return direct children only by default -- If result set exceeds threshold, return summary + suggestion to narrow query -- Use pagination for large search results - -**File Fetching Strategy:** -- Option 1: Keep shallow git clones of indexed revisions -- Option 2: Store file contents in DB during indexing (faster, more storage) -- Leaning toward Option 2 for better MCP performance - -**Revision Indexing:** -- Potentially slow operation (download + parse + index) -- Consider: Separate admin tool vs MCP tool vs async queue -- For initial version: Blocking operation, can optimize later - -**Default Behavior:** -- Should search require specifying revision? -- Could default to "latest indexed" or "nixos-unstable" -- Make configurable - -## Design Decisions - Finalized - -1. **File Storage**: Store file contents in database during indexing - - Better performance for `get_file` tool - - PostgreSQL handles this well - -2. **Default Revision**: Default to configured channel (nixos-stable recommended) - - Conservative approach - stable options likely exist in unstable too - - Configurable via environment variable or config file - -3. **Index as MCP Tool**: `index_revision` is part of MCP server - - Allows Claude to read flake.nix/flake.lock and request indexing - - Use case: "Index the nixpkgs version used by this flake" - - To decide: Blocking vs async (start with blocking, optimize later) - -4. **Testing & Quality** - - Aim for good test coverage across all components - - Write tests alongside implementation (TDD where practical) - - Include benchmarks to measure indexing performance - - Use test fixtures for reproducible testing - - Test both PostgreSQL and SQLite implementations - -## Testing Strategy - -**Unit Testing:** -- Database layer: Test with both real databases and mocks -- Use testcontainers or docker-compose for PostgreSQL tests -- SQLite tests can use in-memory databases -- Mock external dependencies (git operations, network calls) - -**Integration Testing:** -- Use small, curated test fixtures (subset of real options.json) -- Test full indexing pipeline with known data -- Verify search results match expectations -- Test error handling (malformed data, missing revisions, etc.) - -**Benchmarking:** -- Measure time to index full nixpkgs revision -- Track query performance as database grows -- Memory profiling during indexing -- Compare PostgreSQL vs SQLite performance -- Use Go's built-in benchmarking: `go test -bench=.` - -**Test Coverage Goals:** -- Aim for >80% coverage on core logic -- 100% coverage on database operations -- Focus on critical paths and error handling - -## Questions to Resolve - -1. **Revision Auto-Update**: Should we auto-update channel aliases, or manual only? -2. **Indexing Behavior**: Should `index_revision` be blocking or async? - - Blocking: Simpler to implement, Claude waits for completion - - Async: Better UX for slow operations, need status checking - - Recommendation: Start with blocking, add async later if needed +- [ ] Option history/diff - compare options between two revisions ("what changed in services.nginx between 24.05 and 24.11?") +- [ ] Auto-cleanup - prune old revisions after N days or keep only N most recent +- [ ] Man page generation +- [ ] Shell completions (bash, zsh, fish)