Beta9
An open-source, self-hostable runtime for serverless AI workloads that gives a Pythonic interface for fast-starting GPU containers, autoscaling inference endpoints, sandboxed code execution and background task queues.
Beta9 is an open-source runtime for serverless AI workloads, giving a Pythonic interface to deploy and scale AI applications without managing infrastructure. It launches containers in under a second using a custom container runtime, scheduler and embedded caching, fans workloads out to hundreds of containers, supports hot-reloading, webhooks and scheduled jobs, scales workloads to zero by default, mounts distributed storage volumes, and runs on GPUs in the project's own cloud or on GPUs a user brings themselves.
It can spin up isolated sandbox containers to run LLM-generated code, deploy an autoscaling inference endpoint from a plain Python function with a chosen image, GPU, CPU and memory footprint, and schedule resilient background tasks as a replacement for a Celery queue, all through a single decorator-based API.
It is the open-source engine behind a managed cloud platform of the same maker, and can be self-hosted for free instead. It is written in Go under the AGPL-3.0 license.