# OpenAI-Compatible vLLM Serverless Endpoint Worker

A Runpod Serverless worker template that deploys an OpenAI-compatible large language model endpoint powered by vLLM, either from a pre-built Docker image pointed at a model or by building an image with the model baked in.

The recommended path deploys any model with the pre-built `runpod/worker-v1-vllm` Docker image, configured through environment variables such as `MODEL_NAME`, `HF_TOKEN` for gated or private Hugging Face repositories, `MAX_MODEL_LEN` and `QUANTIZATION` (awq, gptq, squeezellm or bitsandbytes); its CUDA compatibility follows the CUDA runtime of the underlying `vllm/vllm-openai` base image used at build time.

An alternative path builds a Docker image with the model weights baked in directly, and a third option runs the official vLLM image as-is for a load-balancing endpoint that answers over direct HTTP instead of a job queue. Served endpoints follow the OpenAI Chat Completions and Responses API shapes, as well as an Anthropic Messages-compatible endpoint, alongside the worker's own standard, non-OpenAI, request and sampling parameters.

Website: https://github.com/runpod-workers/worker-vllm