# Cactus

A hybrid edge-cloud inference engine for running language, vision-language, and speech models directly on mobile devices and wearables.

Cactus provides a quantization pipeline, custom kernels, a runtime, and an inference engine aimed at phones, wearables, smart-home devices, and robots, including a graph mode for composing multiple model calls. Published benchmarks measure decode speed, first-response latency, and RAM use for an LLM (Gemma-4-E2B), a vision-language model, and a speech transcription model (Parakeet-TDT) across Mac and iPhone hardware, alongside accuracy measurements across several quantization bit widths on tasks including ARC, HellaSwag, MMLU, GSM8K, and HumanEval.

A `cactus benchmark` command reproduces these device-side measurements locally, with iOS and Android targets, and a Mac quick-start installs through Homebrew and runs a local demo with one command.

It is written in C++.

Website: https://github.com/cactus-compute/cactus