TypeLLM
A library that adds type-safe, schema-guaranteed output to existing open language models without retraining them, letting a model keep its own free-form reasoning while its final answer stays within a declared JSON schema.
TypeLLM brings type-safe generation to existing autoregressive large language models without changing their architecture or weights. A model keeps its native ability to think freely and generate open text, but its final output is constrained to a declared JSON Schema, so results such as a string, integer, number, boolean or enum choice never fall outside the allowed values.
It reduces the token cost of structured answers with single-token categorical selection and bounded numeric decoding, reuses shared prompt prefixes through key-value caching, and lets independent output fields run in parallel while dependent fields form an explicit execution graph. It also supports an optional thinking mode with a per-field token budget, image input for vision-language models, and permutation averaging to reduce order bias on multiple-choice-style questions.
It is built on SGLang, works with compatible open models a user already serves, and is also offered as an early-access hosted API for people without a GPU.