Inference

A full-featured, multi-workspace AI chat client with local-first storage, real-time cloud sync, a plugin system and MCP support, now deprecated in favor of a successor project.

• Inference

A macOS and Linux terminal UI for managing Ollama models, letting you list, inspect, sort, edit, run, unload, copy, rename, delete and push models with hotkeys.

• Inference
• Terminals

A hardware and configuration guide for running state-of-the-art large language models and speech-to-text locally, covering everything from a $2k single-GPU setup to a $40k multi-GPU rig.

• Inference
• Tutorials

An open-source desktop AI assistant for Linux, Windows and macOS that connects to many model providers and adds agents, RAG, vision, image and video generation, computer use and voice in one chat-style interface.

• Inference
• Generative media

A Windows and macOS desktop app that sends one question to more than a dozen AI chatbots at once, including ChatGPT, Gemini, Claude and Copilot, and shows their answers side by side.

• Inference

A BYOK desktop AI assistant and universal MCP client that connects almost any LLM provider to MCP servers, with chat, image and video generation, RAG over local files, voice mode and system-wide AI commands.

• MCP servers
• Inference

A Swift API for the MLX array framework, letting machine learning research and inference run natively on Apple silicon from Swift, with example apps for chat, text generation and Stable Diffusion image generation.

• Inference
• Libraries

A lightweight C++ inference library for ONNX models built to minimize memory use rather than latency, able to run Stable Diffusion XL on a Raspberry Pi Zero 2 or Mistral 7B on a desktop, with ARM, x86, WebAssembly and RISC-V support.

• Inference

An open-source LLM inference engine for agentic workloads that separates a C++ control plane, which manages request lifecycles and KV cache safety at compile time, from a Python execution plane for fast iteration.

• Inference

A native iOS and Android app for chatting with a self-hosted Open WebUI server, direct OpenAI-compatible and Ollama endpoints, or on-device Apple Intelligence models, built to fix the mobile gaps in Open WebUI's own web client.

• Inference

Node.js bindings for llama.cpp that run language models locally on Metal, CUDA or Vulkan hardware, with pre-built binaries, JSON-schema-constrained output, function calling and embedding support.

• Inference
• Libraries

A cross-platform AI chat client for macOS, Windows, Linux, iOS, Android and the web that connects to Model Context Protocol servers, with local-network data sync between devices.

• MCP servers
• Inference

A Rust ecosystem of crates for running local and remote pre-trained language, audio and image models, with structured generation that constrains model output to a Rust type, regex or custom grammar.

• Inference
• Libraries

Docker images and startup scripts for running the vLLM inference engine on NVIDIA DGX Spark hardware, from a single node to multi-node clusters over InfiniBand or RDMA.

• Inference
• Containers

A modular, full-stack reinforcement learning library for training large language model and vision-language model agents, combining a training framework, a Tinker API backend, an agent layer for long-horizon tasks, and a gymnasium of tool-use environments.

• Models
• Inference

A line-by-line MLX port of several state-of-the-art generative image and video models from the Hugging Face Diffusers and Transformers libraries, letting a Mac run models such as FLUX and Qwen-Image locally through native Apple Silicon acceleration.

• Inference
• Generative media

A self-hostable inference server that turns any computer or edge device into a command center for computer vision: running fine-tuned or foundation vision models, chaining them into workflows, and connecting cameras, video streams and external systems.

• Inference

A highly optimized library of low-level neural network inference operators for ARM, x86, WebAssembly and RISC-V, used as an acceleration backend inside higher-level frameworks such as TensorFlow Lite, PyTorch, ONNX Runtime and MediaPipe rather than directly by model developers.

• Inference

A terminal client for chatting with Ollama, OpenAI, Anthropic and any other provider supported by pydantic-ai, built as a text-based user interface.

• Inference

A native macOS app and command-line tool that runs local language models such as Qwen 3.8 on Apple Silicon using the models' own multi-token prediction heads for faster decoding, exposing an OpenAI- and Anthropic-compatible local server.

• Inference

A JAX-based library for post-training large language models, providing efficient supervised fine-tuning, reinforcement learning and agentic reinforcement learning on TPUs.

• Models
• Inference

A collection of TypeScript libraries for working with the Hugging Face API and Hub: managing repositories, calling more than 100,000 hosted inference models, an MCP client, parsers for the GGUF, DDUF and LeRobot formats, and a small library for building tool-using AI agents.

• Inference
• Libraries

A desktop and server AI assistant application that runs local AI models or connects to mainstream model APIs, with a local knowledge base, web search, agent creation, and the ability to share a configured assistant online for others to use.

• Inference

A Rust interface for hardware-accelerated inference and training on ONNX-format machine learning models, built mainly as a wrapper around Microsoft's ONNX Runtime with support for other pure-Rust runtimes.

• Inference
• Libraries