AIKit
A platform for hosting, deploying, building and fine-tuning open-source large language models, packaging inference and fine-tuning behind an OpenAI-compatible API and OCI-based model distribution.
AIKit offers three main capabilities for open-source LLMs: inference through LocalAI, which exposes an OpenAI-compatible REST API so any OpenAI-compatible client can send requests to the hosted model; fine-tuning through an extensible interface built on Unsloth for a fast, memory-efficient workflow; and OCI packaging, which distributes models as OCI artifacts through any OCI-compliant registry, supporting the CNCF ModelPack specification as well as generic artifact packaging.
It needs no GPU, internet access or extra tools beyond Docker or Podman, runs from a minimal chiseled container image to reduce its attack surface, and supports multi-modal models, image generation, and GGUF and GGML model formats. Declarative configuration drives both inference and fine-tuning, and images can bundle multiple models together.
It runs on AMD64 and ARM64 CPUs, with GPU acceleration through NVIDIA CUDA or AMD ROCm, is deployable to Kubernetes, and its images carry software bills of materials, provenance attestations and signatures for supply-chain security. It also supports air-gapped inference when model content and dependencies are mirrored ahead of time.