MLC LLM
A machine learning compiler and deployment engine that compiles large language models to run across GPUs and platforms including AMD, NVIDIA, Apple and Intel GPUs, web browsers, iOS and Android.
MLC LLM is a machine learning compiler and deployment engine for large language models, aimed at letting developers build, optimize and deploy AI models natively across a wide range of hardware and platforms.
Compiled models run through MLCEngine, a unified LLM inference engine that targets Vulkan and ROCm or CUDA on Linux and Windows, Metal on macOS and iOS, OpenCL on Android GPUs, and WebGPU with WebAssembly in a browser, all backed by the same underlying compiler and engine. MLCEngine exposes an OpenAI-compatible API through a REST server, and through Python, JavaScript, iOS and Android bindings.
It is written in Python and licensed Apache-2.0.