Back to directory

vllm-windows (devnen patched fork)

A patched, native-Windows build of vLLM adding three Windows-specific fixes, a Qwen3 reasoning parser and tool-calling backports, published as prebuilt wheels for both Ampere/Ada and Blackwell RTX GPUs.

vllm-windows (devnen patched fork)

This is a thin patch series on top of SystemPanic's existing community Windows build of vLLM (base version 0.19.0), adding fixes that make real-world inference and agentic workloads work correctly on Windows: a CPU relay for Gloo, a Qwen3 reasoning parser, and support for a wildcard model name, plus Qwen3.5/3.6 tool-calling backports.

Two wheels are published: one for Ampere/Ada GPUs (RTX 30/40 series) built against vLLM 0.19.0 and CUDA 12.6, and a separate one for Blackwell GPUs (RTX 50 series) built against vLLM 0.20.0 and CUDA 13.2, since Blackwell's sm_120 kernels require the newer toolchain and driver; the older wheel will load on a Blackwell card but its kernels will not dispatch.

The repository provides the source tree, patch backups, the prebuilt wheels and a matching chat template; end users who just want a one-click way to run a model are pointed instead to a companion launcher repository that bundles this engine.

More in Inference

See all 770
  • OpenDecision

    deepanwadhwa.github.io

    An open-source Python package that answers typed questions about application state or documents using a local natural-language-inference model, returning structured choice, score, yes/no or evidence-relation values instead of free text.

  • Lifeboat

    iterate.ai

    A local inference server that puts an OpenAI- and Anthropic-compatible API in front of open-weight language models running on hardware you control, from a laptop to a GPU cluster.

  • Claude S40

    github.com

    An unofficial Claude chat client for 2007-era Nokia Series 40 phones, built as a Java ME app plus a small Go relay server, with web search, reading mode and saved chats.