# swift-litert-lm

A Swift package that runs Google's Gemma 4 model on iPhone through the LiteRT-LM runtime on the Metal GPU, with text, image and audio input, and a second mode that plugs the same runtime into Apple's Foundation Models API as a custom backend.

swift-litert-lm puts two interfaces on top of Google's official LiteRT-LM runtime. Its "easy mode" is a single `LiteRTChat(.gemma4_E2B)` call that downloads the model on first run and streams a Gemma 4 chat response token by token on the Metal GPU, measured at roughly 50 tokens per second on an iPhone 17 Pro.

Its "FM mode" instead exposes `LiteRTLanguageModel` as a backend for Apple's Foundation Models API, introduced in iOS 27 to accept third-party models conforming to its `LanguageModel` protocol; through a `LanguageModelSession`, it supports the same `respond`/`streamResponse` calls, `@Generable` guided generation, and tool calling as Apple's own on-device and Private Cloud Compute models, along with image, audio and video understanding that Apple's system model does not offer.

It installs as a single Swift Package Manager dependency and requires no other setup beyond adding the package and calling one of these two entry points.

Website: https://github.com/john-rocky/swift-litert-lm