llama_sdk
A Dart implementation of llama.cpp that lets Flutter and Dart applications run local language models, used by the Maid mobile AI app.
Visit website (opens in a new tab)github.com
llama_sdk is a Dart package that wraps llama.cpp, letting Dart and Flutter applications load and run local large language models. It is the inference layer used by the Maid mobile artificial intelligence distribution.
The package supports isolates for parallel processing and streams generated tokens as they are produced. It is installed as a pubspec.yaml dependency, and a model is loaded by pointing a Llama instance at a local model file with configurable context and batch sizes, then sending chat messages and reading the response as a stream.
It is released under the MIT license.