# Rllama

Ruby bindings for llama.cpp that run open-source language models such as Gemma, Qwen, Llama and GPT-OSS locally, directly inside a Ruby application, plus a standalone interactive CLI chat.

Installed as the rllama gem, running the rllama command with no arguments lists already-downloaded models under a local models directory and a curated list of popular ones -- Gemma, Qwen, Nemotron, LFM2, GLM, GPT-OSS, Llama and Phi among them -- that download automatically from Hugging Face on first use; a model path or URL can also be passed directly.

The library API generates text completions, with sampling parameters read by default from a model's own GGUF metadata and falling back to llama.cpp's standard defaults, reporting back which values were actually used. It supports token-by-token streaming generation and an optional system prompt to guide model behavior, and is licensed MIT.

Website: https://www.docuseal.com/blog/run-open-source-llms-locally-with-ruby