SimLLM
A network-faithful simulator that predicts the serving performance of large LLM deployments before you buy or reserve hardware, modeling the fabric at packet level rather than as a simple bandwidth estimate.
SimLLM predicts serving metrics such as time-to-first-token, time-per-output-token, goodput, and SLO attainment for large LLM deployments, using a discrete-event, packet-level network simulator underneath rather than a bytes-over-bandwidth estimate. At scale, incast, queue buildup, congestion-control oscillation, and head-of-line blocking reshape latency distributions in ways a simple analytic model misses, so SimLLM couples the real frontend scheduler of a serving framework with a simulated GPU executor and a packet-level network backend, letting scheduling, KV and prefix-cache management, and the network fabric feed back on each other.
It models GPUs and NICs uniformly as devices with typed ports, connected by PCIe inside a machine, NVLink or xGMI between GPUs, and Ethernet across the cluster, and simulates every scheduler step's collective traffic, such as tensor-parallel all-reduce or MoE all-to-all, at packet granularity over a Clos fabric. It integrates with existing serving frameworks without forking them: vLLM plugs in through its distributed-executor-backend flag, and SGLang through its own plugin framework.
It is developed in the open and sponsored by OpenFabric, a company that designs network hardware for distributed AI workloads.