# plow

An LLM inference compiler and runtime that compiles a Hugging Face checkpoint into a static packet stream and executes it with a persistent, resident GPU interpreter, avoiding per-operation CPU dispatch, with an OpenAI-compatible HTTP server.

plow ahead-of-time compiles a Hugging Face model checkpoint into a packet stream with an AOT compiler, plowc, then runs it with a persistent on-device interpreter that stays resident on the GPU as one cooperative kernel launch, executing operations at warp granularity and coordinating through counters rather than dispatching each operation from the CPU. A host runtime, plowrt, serves the compiled model over an OpenAI-compatible HTTP API, and CUDA and HSA interpreters back NVIDIA and AMD GPUs respectively; a Lean 4 component checks rewrite rules and the counter protocol formally.

Fully supported hardware today includes the NVIDIA RTX 5090 and RTX PRO 6000 Blackwell, and the AMD Instinct MI350X, MI355X and MI300X, each with GPU-specific compiler flags and environment variables; a compiled packet emit is tied to a specific SM or CU count and is not portable across different GPU models.

Documentation covers the compiler pipeline, packet ABI, counter system, cost model and a staged playbook for bringing up a new model on the runtime.

Website: https://infervisor.ai/plow