libargus
An unmanaged, zero-allocation native AI inference runtime for Java 22 and newer that consolidates vision, speech and LLM pipelines behind one native memory boundary.
libargus is an unmanaged, zero-allocation native AI execution runtime that consolidates LLM text generation, Whisper-based speech-to-text, speech-LLM text-to-speech and multimodal vision, audio and video encoding and evaluation pipelines into one process-global native runtime, built directly on the GGML and llama.cpp compute engines and exposed through a single Project Panama Foreign Function and Memory boundary for Java 22 and newer.
It provides a unified, thread-safe C API designed for zero-copy compilation alongside unmanaged orchestration frameworks. Recent additions include model-topology introspection that detects recurrent, hybrid or diffusion model types to prevent unsafe operations, zero-rollback KV-cache sequence-slot forking with full sampler-state cloning for token history and randomness continuity, and dedicated clone-slot pooling that reuses a unified KV cache to avoid extra VRAM allocation, alongside strict structural validation of slot operations.
It is distributed on Maven Central, with a core artifact plus separate native-runtime provider artifacts published per target platform.