# vLLM Factory

A set of vLLM plugins that serve encoder, pooler and structured-prediction models such as ColBERT, GLiNER and ColPali for production inference, without forking vLLM.

vLLM Factory packages encoder-style models as vLLM plugins so they can be served the same way as generative LLMs, for both standalone and in-process deployment. It ships 12 encoder plugins covering retrieval (ColBERT variants), named-entity recognition and linking (GLiNER), and multimodal retrieval (ColPali, ColLFM2), along with pre- and post-processing and continuous batching.

Its benchmarks compare vLLM Factory against vanilla PyTorch on an NVIDIA RTX A5000, reporting throughput gains from roughly 1.7x to 12.6x depending on the model, with output checked against a reference implementation (cosine similarity for embeddings, entity recall for NER). A beta multi-instance mode runs several vLLM instances on one GPU behind a shared async dispatcher, since encoder workloads are usually memory-bound rather than compute-bound.

It installs as a Python package alongside vLLM 0.19 or later, and is served with the standard vLLM serve command plus an IOProcessor plugin flag.

Website: https://github.com/ddickmann/vllm-factory