Deka
A human-in-the-loop workbench that turns a domain expert's intuitive sense of "find me content like this" into a reproducible, auditable classifier trained on a rubric-judged sample of a large text corpus.
Deka escalates a domain expert's tacit judgment about a query into a precise, reproducible labeling process over a corpus too large to read by hand, for questions that are concept-shaped rather than expressible as a keyword filter or a factual lookup. It proceeds through four phases: probing hybrid dense and learned-sparse retrieval until an operator's relevance judgments converge, harvesting the embedding neighborhood of validated examples, refining that cohort into an explicit, auditable rubric judged by a language model on a stratified sample, and applying a low-cost classifier trained on that judged sample to label the full corpus.
A finished session produces a self-contained labeled dataset that can be analyzed without access to Deka's own infrastructure, and a persisted logistic-regression classifier that can label future data for the same concept without further language-model cost. The project is explicitly scoped to cases where relevance cannot be reduced to a keyword search or answered by retrieval-augmented generation alone.