# WeeLLM

A layer-streaming inference tool that runs large diffusion image, image-editing and video-generation models on GPUs with as little as 4 GB of VRAM, without quantizing the model weights.

WeeLLM dynamically determines how many layers of a diffusion model's text encoder and transformer fit in the available VRAM, pins that many blocks on the GPU, and streams the remaining layers from disk in the background layer by layer, so models that would otherwise require far more memory can still run at full precision. It supports a wide range of image-generation and image-editing model families, including several Stable Diffusion versions, Flux variants, Qwen Image, HiDream and others, plus two text-to-video and image-to-video models.

Running it requires explicitly passing the available VRAM and RAM budgets to the command line so it stays within hardware limits; it is optimized for local machines with fast NVMe storage and modern GPUs, and the project notes it will be slower on typical cloud instances with restricted disk speed.

It is written in Python, installable from source or with pip, and runnable in Kaggle or Colab notebooks.

Website: https://github.com/Jit-Roy/WeeLLM