Back to directory

paddock

A command-line tool that tells you which local LLMs fit an Apple Silicon Mac, how fast they will run, and how to launch them, in one command.

paddock

paddock answers three questions that picking a local model normally means working out by hand: whether a model fits, given that macOS caps how much memory the GPU can actually use; how fast it will run, since a model that only reaches a few tokens per second is not usable; and how to launch it, replacing manual Ollama tags or llama-server flag combinations.

It reads macOS's real Metal working-set limit directly to budget whether a model fits fully on the GPU, and ships a per-Apple-chip memory-bandwidth table from the M1 through the M5 generation, since Apple Silicon token generation is memory-bandwidth bound and bandwidth varies widely across the lineup. Speed estimates model per-token traffic as active weights plus an 8000-token-deep key-value cache, so they reflect real long-context decoding rather than an empty-context best case.

It treats Apple's own MLX framework as a peer of Ollama and llama.cpp, and running it with no arguments opens an interactive terminal interface that loads instantly from the last catalog snapshot and refreshes in the background.

More in Inference

See all 770
  • OpenDecision

    deepanwadhwa.github.io

    An open-source Python package that answers typed questions about application state or documents using a local natural-language-inference model, returning structured choice, score, yes/no or evidence-relation values instead of free text.

  • Lifeboat

    iterate.ai

    A local inference server that puts an OpenAI- and Anthropic-compatible API in front of open-weight language models running on hardware you control, from a laptop to a GPU cluster.

  • Claude S40

    github.com

    An unofficial Claude chat client for 2007-era Nokia Series 40 phones, built as a Java ME app plus a small Go relay server, with web search, reading mode and saved chats.