GranitePi-4-Nano
A beginner-friendly guide and automated setup script for running the 350-million-parameter IBM Granite 4.0 language model fully offline on a Raspberry Pi 5 through Ollama, using a GGUF quantization tuned for the Pi 5's CPU.
This repository is a complete, beginner-friendly guide to setting up IBM Granite 4.0 (350M) on a Raspberry Pi 5 with Ollama for fully local, private inference with no cloud dependency; it uses a specific GGUF quantization, unsloth_granite-4.0-h-350m-GGUF, tuned with particular thread settings for smooth performance on the Pi 5's CPU.
An automated setup script checks the hardware, RAM, disk space and thermals, installs the needed dependencies, and pulls the model so the user can start chatting immediately. The hybrid Mamba-2 model is about 366 MB to download and needs roughly 800 MB to 1.2 GB of RAM at inference, supports more than 12 languages, and works fully offline once set up.
Requires a Raspberry Pi 5 with 8 GB RAM recommended, 4 GB supported with warnings, at least 5 GB free storage, active cooling to avoid thermal throttling, and 64-bit Raspberry Pi OS Bookworm or later. Apache-2.0 licensed model and script.