# Emmy

A GPU compiler and benchmarking stack that optimizes a large language model for inference through kernel fusion, autotuning and scheduling, then deploys and tracks the results across GPU types.

Emmy compiles, benchmarks and deploys large language models for inference on GPUs. Its compiler traces an ordinary PyTorch module, fuses it into a single kernel through kernel fusion, autotuning and scheduling, runs it, and lets the developer inspect the emitted CUDA code, aiming to outperform existing engines on specific models.

Installing the command-line tool pulls in a set of recommended, already-optimized recipes; a separate extra installs the compiler itself along with PyTorch, transformers and a C++ interoperability layer, and building the project from source additionally needs a Rust toolchain, since its runtime extension is written in Rust. Kernels compile through the CUDA toolkit's own compiler and launch through that Rust runtime. Commands that only drive remote hardware, such as deploying, benchmarking or tearing down a virtual machine, do not need the local toolchain at all.

Website: https://github.com/cloudrift-ai/emmy