# vllm-swift

A native Swift and Metal backend plugin for vLLM on Apple Silicon that replaces the Python inference hot path, serving an OpenAI-compatible API with faster short-context decoding than the existing Python and MLX engine.

vllm-swift is a drop-in plugin that lets vLLM run its inference hot path through a native Swift and Metal bridge on Apple Silicon instead of through Python and MLX, while keeping vLLM's own serving flags and OpenAI-compatible API unchanged. It installs through Homebrew, which bundles a prebuilt bridge and Metal kernel library needing no compilation, through pip, which bundles the same prebuilt pieces, or from source with a provided install script.

Measured on an M5 Max with 128GB of memory, the project reports decode throughput up to roughly 2.6 times faster than the existing Python and MLX vLLM engine on short-context, single-request generation, with a smaller but still favorable gap at higher concurrency; full benchmark methodology and long-context results are documented separately. A KV-cache compression feature can trade some throughput for a longer usable context at a given memory budget.

Website: https://github.com/TheTom/vllm-swift