local-llm-lab
A CLI for planning whether a large language model will fit and run well on a given machine's unified memory or VRAM, before downloading the weights, with dry-run deployment scripts and benchmarking.
local-llm-lab answers whether a given model will fit and run acceptably on a specific machine before a user spends time downloading its weights. It detects local hardware, such as the OS, CPU and GPU, unified memory or VRAM, disk, and available backends like Metal, CUDA, AVX or NEON, saves the resulting hardware profile for reuse, and plans whether a model fits by working through weights, key-value cache, overhead, OS reserve and margin, producing a verdict, a risk level and a recommended backend and quantization.
It can generate llama.cpp, MLX and Ollama dry-run scripts and configuration with a preflight checklist without downloading the actual model, run benchmarks using deterministic mock data or small local smoke checks, simulate stress from resource contention alongside another GPU workload, and produce reports in Markdown, JSON, SVG charts or a static HTML page served locally.
It is a Python command-line tool built for developers with 64 to 256 GB of unified memory, particularly Apple Silicon Max and Ultra Macs, with Linux and NVIDIA support planned. The core CLI needs no third-party dependencies.