Kalosm
A Rust ecosystem of crates for running local and remote pre-trained language, audio and image models, with structured generation that constrains model output to a Rust type, regex or custom grammar.
Kalosm is an ecosystem of Rust crates for developing applications around pre-trained AI models. The core Kalosm crate provides a simple interface to models such as Llama, Mistral and Phi for text, Whisper for audio transcription, Segment Anything for image segmentation and Bert for text embeddings, each available quantized and with GPU acceleration. A companion project, Fusor, is a compiler-backed runtime for quantized ML inference on CPUs and WebGPU accelerators.
Beyond running models, it includes utilities for extracting and chunking context from text, HTML, DOCX, Markdown and PDF files with vector database integration for semantic search, transcribing audio from a microphone or file, and crawling and scraping web pages. Its structured generation feature uses a custom parser engine and sampler so a Rust type annotated with derive macros, a regex, or a hand-written grammar can constrain a model's output to valid JSON, HTML, XML or other structured formats, faster than unconstrained generation.
It is written in Rust and released under the Apache-2.0 license.