Runpod serverless runner for ollama
This project packages Ollama as a serverless worker for the RunPod.io platform, downloading a named model automatically on startup and serving both native Ollama and OpenAI-compatible streaming requests.
A RunPod serverless endpoint is started from a published Docker image, with an environment variable naming the Ollama model to download from ollama.com automatically; a mounted volume is used automatically when available so a downloaded model can persist between runs. Streaming responses are supported for OpenAI-compatible requests, and the repository includes example test request payloads for use in the RunPod console, along with instructions for preloading a model directly into the Docker image instead of downloading it at startup.
It is a thin deployment wrapper around Ollama rather than a new inference engine, aimed at someone who wants Ollama available as pay-per-use serverless GPU capacity rather than a machine they keep running.
It is written in Python and released under the Creative Commons Attribution 4.0 license, requiring attribution back to the original repository.