lightmetal
A zero-dependency Java library and command-line tool that runs GPU-accelerated large-language-model inference on Apple Silicon from a single executable JAR, by binding a Metal-enabled llama.cpp build through Java's foreign-function interface.
lightmetal runs GGUF-format language models on the GPU of an Apple Silicon Mac from a single Java 25 executable JAR with no other dependencies, by binding a Metal-enabled libllama shared library through Java's Foreign Function and Memory API. It supports Mistral, Gemma, Nemotron and Mellum architecture models, including specific named releases of each.
It runs as a one-shot command that generates a completion for a prompt with configurable sampling options, or with a serve flag it starts an Anthropic-compatible messages endpoint that keeps the model loaded and serializes requests since llama.cpp contexts are not thread-safe. An install script fetches a prebuilt JAR and helper scripts, or the JAR can be fetched directly or built from source, and the model to load can be set once in a properties file so later invocations only need to supply a prompt.