Spezi LLM
A Swift package that adds large-language-model capabilities to apps built on the Spezi framework, covering local on-device inference, OpenAI-hosted models, and inference offloaded to nearby Fog nodes.
Spezi LLM is a set of Swift package targets for integrating LLM functionality into Spezi-based applications. SpeziLLM provides the base infrastructure shared by the other targets; SpeziLLMLocal runs open-source models such as Llama 2, Phi or Gemma directly on-device with a Swift API and no low-level code required; SpeziLLMLocalDownload manages downloading and storing local models, including onboarding views; SpeziLLMOpenAI integrates with OpenAI's hosted GPT models through its API; and SpeziLLMFog discovers and dispatches inference jobs to Fog node resources on the local network.
SpeziLLM Local is not compatible with simulators, since the underlying mlx-swift dependency needs a Metal GPU that simulators do not provide. Documentation for each target is provided through DocC.