code-llm-completion
A training and serving stack for fine-tuning open code models such as StarCoder2 and DeepSeek-Coder into low-latency IDE completion models, aimed at low-resource languages and on-premises deployment that mainstream hosted copilots skip.
code-llm-completion is a focused pipeline for turning open code LLMs - StarCoder2-7B and DeepSeek-Coder-6.7B in the examples shown - into fast, IDE-grade completion models, with particular attention to languages that large hosted copilots cover poorly, such as Indonesian PHP and Laravel code or Lua game scripts. It prioritizes completion latency, suggestion acceptance rate and simple on-premises deployment, in that order, aiming for an OpenAI-compatible endpoint any editor extension can call after a single serving command.
The stack uses Hugging Face transformers and Trainer for fine-tuning, DeepSpeed ZeRO-3 with CPU offload to fit a 7B model and its LoRA adapters on a single AMD MI250X GPU, PEFT for LoRA and bitsandbytes for 4-bit quantization, and vLLM for serving behind a thin OpenAI-compatible shim. All entry points are plain argparse CLIs in bin/ with no hidden YAML-only configuration layer.
It targets AMD ROCm hardware specifically, framed as a lower-cost alternative to renting NVIDIA GPU clusters for on-premises code-completion fine-tuning.