Model Serving Minefield
A public, symptom-first registry of LLM serving-path bugs that silently produce wrong benchmark numbers, covering chat templates, tool parsers, quantization kernels and eval harnesses, each entry with a reproduction status and the check that catches it.
Model Serving Minefield is a public registry of serving and configuration bugs that produce a confidently wrong measurement on a real LLM serving path even though the request and response both look correct. Each entry leads with the symptom someone would actually observe, then explains the underlying mechanism, which stacks and builds are affected, the check that catches it, and the fix.
Every entry carries exactly one status from a closed vocabulary describing how much weight it carries: reproduced by the maintainers, contributor-measured with reported conditions, reported by others without independent reproduction, measured but with raw data not published, or still under test; entries are labelled rather than rejected so readers can tell how much to trust each one.
A companion Python package installs a bounded, read-only command-line doctor that can check a live OpenAI-compatible endpoint, and the repository includes files meant to be handed directly to an AI agent to help diagnose a serving problem against the registry.