tpu-inference
The TPU inference backend for vLLM, unifying JAX and PyTorch model execution under a single lowering path so PyTorch models can run performantly on TPU hardware without extra code changes.
tpu-inference is the hardware plugin now powering vLLM TPU, replacing the previous backend with a unified path for both JAX and PyTorch model definitions on Google TPUs. It aims to extend TPU performance in the open, extend native support to JAX models, and keep vLLM's standard user experience, telemetry and interface unchanged for users switching hardware.
The project publishes recommended models and features that have passed unit, integration and performance testing, since not every model supported elsewhere in vLLM is yet validated on this backend, alongside a live support-matrix dashboard covering models, features and kernels across recommended (v7x, v5e, v6e) and experimental (v3, v4, v5p) TPU generations.