vLLM MBLT
An out-of-tree vLLM plugin that adds Mobilint NPU support, so Mobilint-optimized language and vision-language models can be served through ordinary vLLM commands.
vllm-mblt is an out-of-tree vLLM plugin that integrates Mobilint NPU runtime support into the vLLM serving and benchmarking stack, registering a custom platform, worker and model registry hooks without patching vLLM itself. This lets Mobilint-optimized artifacts be served through familiar vllm serve commands and OpenAI-compatible APIs.
It supports Mobilint wrappers for the Llama, HyperCLOVAX, EXAONE/EXAONE4 and Qwen2/3 and Qwen2/3-VL model families, reads model-configured prefill chunk size and batch size to tune chunked prefill and scheduler concurrency automatically, and is compatible with vllm serve, vllm bench serve and vllm bench throughput. It requires a Mobilint NPU environment and a pinned vLLM version for compatibility.
It is written in Python and Apache-2.0 licensed.