GenAI on EKS Starter Kit
A starter kit that deploys production-ready generative-AI infrastructure on Amazon EKS with a single CLI command, including LLM serving engines, an AI gateway, vector databases and observability.
The kit's configurable components span an AI gateway such as LiteLLM or Kong AI Gateway OSS, LLM serving through vLLM, SGLang or Ollama, an embedding model server using Text Embedding Inference, observability through Langfuse or Phoenix, a GUI through Open WebUI, vector databases including Qdrant, Chroma or Milvus, workflow automation with n8n, an AI agent, an MCP server, and agent frameworks such as Strands Agents or Agno.
It requires the AWS CLI, Terraform, Node.js, kubectl, Helm and Docker with Buildx already installed. A quick demo setup provisions the infrastructure with Terraform and deploys the example components from a configuration file in order, while an interactive setup instead prompts for which components to install and for values such as the AWS region, the EKS cluster name and a domain name. It is licensed MIT-0.