# ModelForge

A one-stop NLP platform for small teams that trains, evaluates, and deploys small encoder models such as BERT and BGE, and separately runs blind, human-and-AI-scored prompt evaluations for large model prompts, without fine-tuning large models itself.

ModelForge targets a specific gap for small teams: many practical NLP tasks such as sentiment analysis, intent classification, text classification, named entity recognition, and semantic similarity are better served by a small, fast, locally deployable encoder model than an expensive large-model API call, but building the full pipeline of data versioning, training, evaluation, deployment, and error-driven retraining from scratch is slow. It manages dataset and evaluation-set versions as immutable, checksummed snapshots, runs classification, NER, sentence-pair, and embedding training recipes tracked in MLflow, and deploys a chosen model version behind prediction, embedding, and similarity endpoints.

Production errors reported back through an authenticated API endpoint are triaged in a labeling workbench and can be folded back into a new training set to close the error-correction loop, with an automatic score showing how many previously broken cases a retrain fixed. Separately, for teams that also use large-model APIs, it runs blind prompt evaluations, comparing prompts or models with both human judges and an LLM-as-judge under a hidden-identity, bias-resistant scoring flow, without ever fine-tuning the large model itself.

Website: https://github.com/LingyiChen-AI/ModelForge