Foundry Local
Microsoft's on-device AI inference solution for running generative AI models locally through a CLI, SDK or REST API; this repository holds tutorials and examples for it.
Foundry Local runs generative AI models directly on local CPUs, GPUs or NPUs, keeping prompts and outputs entirely on the device for privacy and low latency, and working fully offline once models are downloaded. It exposes an OpenAI-compatible REST API so the same client code can target local or Azure-hosted endpoints with minimal changes, alongside CLI, Python, JavaScript and .NET SDKs.
It automatically detects the host's hardware and downloads the best-optimized model variant for NVIDIA CUDA, AMD DirectML, Apple Metal, Intel or Qualcomm NPUs, or CPU. It runs entirely standalone with no Azure subscription required, while also supporting hybrid cloud-to-edge workflows with Azure AI Foundry.
This repository is a collection of Jupyter notebook tutorials, practical examples and reference material for building with Foundry Local, which supports Windows 10 and 11 (x64 and ARM), Windows Server 2025, and macOS on Apple Silicon, and is currently in preview.