BARGAIN
A framework that lowers the cost of processing a dataset with large language models by deciding per record whether a cheap or an expensive model is used, guaranteeing a target accuracy.
BARGAIN reduces the cost of running a large language model over a dataset by following a model-cascade approach: for each record, it first runs a cheap model and checks the model's own output probability against a threshold to decide whether that answer can be trusted. When it cannot, it falls back to a more expensive, more accurate model, maximizing how often the cheap model is used while meeting a user-specified accuracy target.
The cascade threshold is estimated in a preprocessing step by sampling and labeling a small number of records, which the project says gives BARGAIN a guarantee that holds for any sample size, unlike related systems such as FrugalGPT, which give no guarantee, or SUPG, whose guarantee only holds asymptotically. The method is described in a SIGMOD 2026 paper, and the repository includes example tasks such as classification and extraction over real-world datasets.