SwiftLM
A native Swift inference server that serves MLX language models on Apple Silicon through a strict OpenAI-compatible API, without a Python runtime, and can stream oversized mixture-of-experts models directly from disk.
SwiftLM is a native Swift inference server that serves MLX models on Apple Silicon through an OpenAI-compatible API, avoiding a Python runtime, its global interpreter lock, and unnecessary memory copies in favor of a single compiled binary. It runs from a pre-built binary that bundles its Metal kernel library, or can be built from source, which also compiles the Metal kernels and the Swift binary itself.
An option to stream expert layers directly from NVMe storage lets it run mixture-of-experts models larger than available unified memory without triggering macOS virtual-memory swapping. The project publishes detailed benchmark tables of decode speed, longest supported prompt length, and peak GPU memory for specific model and hardware combinations, including results from a lower-memory Mac mini that surfaced and fixed memory bugs hidden on machines with more RAM.
It is written in Swift and distributed as a downloadable release binary or built from source with a provided build script.