Back to directory

vllm-gb10

A reproducible vLLM Docker image built specifically for the NVIDIA DGX Spark's GB10 chip, with every component in the stack pinned by commit or digest so the same version file always rebuilds the same image.

vllm-gb10

vllm-gb10 packages vLLM for the NVIDIA DGX Spark (GB10 SoC, sm_121a) as an arm64 Docker image, targeting that hardware only; it will not run on x86 machines or other GPU architectures. Every component of the stack, including the CUDA base image, PyTorch, NCCL and FlashInfer, is pinned by commit SHA or digest in a versions.env file and rebuilt from source where noted, with matching lockfiles under locks/.

A Rust frontend (vllm-rs) built on axum and PyO3 sits alongside the Python stack, and companion CUDA packages such as NVSHMEM, TVM-FFI and TileLang are pinned to exact versions.

Each release publishes four tags: a canonical immutable tag tied to the vLLM version and stack revision, a variant that also names the CUDA and PyTorch versions, a mutable latest tag, and a tag tied to the exact commit that produced the build; there is no bare, mutable version tag.

More in Inference

See all 770
  • OpenDecision

    deepanwadhwa.github.io

    An open-source Python package that answers typed questions about application state or documents using a local natural-language-inference model, returning structured choice, score, yes/no or evidence-relation values instead of free text.

  • Lifeboat

    iterate.ai

    A local inference server that puts an OpenAI- and Anthropic-compatible API in front of open-weight language models running on hardware you control, from a laptop to a GPU cluster.

  • Claude S40

    github.com

    An unofficial Claude chat client for 2007-era Nokia Series 40 phones, built as a Java ME app plus a small Go relay server, with web search, reading mode and saved chats.