imp
An LLM inference engine built from scratch in C++ and CUDA for a single GPU model, the NVIDIA RTX 5090, serving models over both OpenAI and Anthropic-style APIs.
imp is an LLM inference engine that targets exactly one chip, the NVIDIA RTX 5090 and its close Blackwell siblings, rather than aiming for broad hardware support. It is written from scratch in C++23 and CUDA, with its own GGUF and SafeTensors loaders, tokenizer, paged KV cache and kernels, and it serves models over both the OpenAI and Anthropic APIs natively.
It is also usable as a C library and CLI, not just a server, and ships with a Docker-based quickstart that runs a quantized 27B model on a single 32 GB card. It has no CPU path and does not support multiple GPUs, and the project documents who should use llama.cpp or vLLM instead. It is MIT licensed.