# CrossPool

A serving system that lets multiple SGLang model instances share GPU memory by separating KV-cache capacity from FFN weight execution, so each model's resource needs no longer have to be reserved together.

CrossPool co-locates multiple SGLang models on shared GPUs when a model's KV-cache demand and its FFN weight requirements do not scale together. A conventional co-located deployment ties both resources to the same process and GPU reservation, forcing every instance to reserve both even when one mainly needs KV capacity and another mainly holds resident weights.

A daemon control plane registers participants, coordinates SLO-aware KV capacity, and tracks liveness. Each SGLang instance keeps logical KV and prefix-cache ownership while sending FFN work to shared agents that hold model-specific weight shards as reusable graph templates replayed through per-lane CUDA graphs; physical KV backing can move between co-located instances without sharing logical KV contents.

It installs as a pinned SGLang plugin that adds architecture-specific adapters without replacing SGLang's own request or KV-cache runtime, and is written in Python.

Website: https://github.com/Coekjan/CrossPool