LiteRtLmSharp
Unofficial .NET bindings for Google's LiteRT-LM, running on-device LLM inference such as Gemma from any .NET application on Windows, Linux, Android, macOS or MAUI, without a server or the cloud.
The bindings call LiteRT-LM's C API through P/Invoke, with native binaries distributed per runtime identifier as NuGet packages in the style of LLamaSharp; CPU and GPU backends are validated on real hardware for Windows, Linux and Android, with macOS GPU currently validated in CI over the WebGPU/Dawn path.
Supported features include blocking, awaitable and streaming chat; function calling with constrained decoding for reliable JSON arguments; Gemma's separate "thinking" reasoning output; multimodal image and audio attachments; persisting and cloning conversation state; tokenization and context-window budgeting; and speculative decoding with a benchmark API.
It integrates with the .NET AI ecosystem through IChatClient and Semantic Kernel connectors, and is source-generated for Native AOT and trimming compatibility.