AnyJev
A tool that converts an existing open large language model into a calibrated decision model, returning a typed answer and a real probability for a choice, a yes/no or a score, without any additional training.
AnyJev turns an open LLM into a Jev-style decision model. Given a typed question, such as a choice among options, a yes/no, or a score, it returns a decision along with a calibrated probability, using the model's own hidden states rather than free-form generation.
It works by truncating a model to roughly the blocks a decision needs, usually about two-thirds of the network, then serving it with vLLM as an embedding endpoint whose pooler returns the hidden state a small closed-form head reads; a single pipeline command converts, serves and measures a model end to end, reporting accuracy, calibration error and latency per decision against held-out data. The authors report that cutting a model down this way can leave accuracy unchanged or slightly higher while being faster, since a middle layer can be a better feature space for a linear head than the final layer.
It is a Python package, developed by researchers at Nokia and Tencent, and is inspired by TypeSafe AI's Jev.