# InferCrane

Open-source infrastructure for running self-hosted language models in production, deploying vLLM, SGLang or a custom backend across AWS, GCP, Kubernetes or RunPod behind one stable OpenAI-compatible endpoint with guarded, evidence-based releases.

InferCrane manages the full lifecycle of a self-hosted model deployment behind a single OpenAI-compatible endpoint, so the calling application never has to learn about changes to the underlying serving topology. It can deploy vLLM, SGLang or a custom OCI workload, or adopt an already-running compatible endpoint, across AWS, GCP, Kubernetes and RunPod.

A release loop evaluates a new model or configuration as an isolated candidate against benchmark, replay, quality, reliability and cost evidence before deciding to accept, reject or wait on promoting it to production traffic; a passing health check alone is not enough to move traffic. Long-running operations persist their intent before mutating a cloud provider, so a disconnected CLI or worker can reattach to the same operation instead of losing track of what completed. It installs as a CLI through Homebrew, with matching Python and TypeScript SDKs and a Terraform provider, and includes a GPU-free demo that proves the release loop without creating cloud resources.

Website: https://infercrane.com/