swift-litert-lm
A Swift package that runs Google's Gemma 4 model on iPhone through the LiteRT-LM runtime on the Metal GPU, with text, image and audio input, and a second mode that plugs the same runtime into Apple's Foundation Models API as a custom backend.
swift-litert-lm puts two interfaces on top of Google's official LiteRT-LM runtime. Its "easy mode" is a single LiteRTChat(.gemma4_E2B) call that downloads the model on first run and streams a Gemma 4 chat response token by token on the Metal GPU, measured at roughly 50 tokens per second on an iPhone 17 Pro.
Its "FM mode" instead exposes LiteRTLanguageModel as a backend for Apple's Foundation Models API, introduced in iOS 27 to accept third-party models conforming to its LanguageModel protocol; through a LanguageModelSession, it supports the same respond/streamResponse calls, @Generable guided generation, and tool calling as Apple's own on-device and Private Cloud Compute models, along with image, audio and video understanding that Apple's system model does not offer.
It installs as a single Swift Package Manager dependency and requires no other setup beyond adding the package and calling one of these two entry points.