Back to directory

ENGRAFT

A method that writes new facts directly into the n-gram lookup table some recent language models carry alongside their transformer, as a small removable overlay file, without changing any of the model's trained weights.

ENGRAFT

Some recent models carry a large n-gram lookup table next to the transformer, which DeepSeek calls Engram and which llama.cpp calls the PLE table in Qwen3.8-Flash-Next; it is read at every position by hashing the last few tokens and adding the result to the residual stream early in the model's computation. ENGRAFT trains new rows into that table while leaving every other weight of the model untouched, producing a single small overlay file that a llama.cpp fork applies at read time; removing the overlay restores the model exactly, and the underlying GGUF file on disk is never modified.

Because a grafted fact is keyed to specific token n-grams, it only fires when a prompt contains those n-grams, and the project's own measurements show that a fact written in one language does not transfer to a query in another. In exchange, a grafted fact costs no context tokens and needs no retriever, index or second model at inference time.

Every measurement in the repository was made on Qwen3.8-Flash-Next; DeepSeek V4.1 Flash is described as a future target that is not yet supported. It is written in Python under the Apache-2.0 license, with a companion website presenting the method and its measurements.

More in Models

See all 405
  • mini-AGI

    github.com

    A continual learning language model that trains from scratch on a single 8 GB VRAM GPU, assembling its own architecture and adapting its size to the hardware and data available rather than shipping as a fixed pretrained model.

  • An open-source contrastive-embedding pipeline that maps technical CVE vulnerability descriptions to formal cybersecurity compliance controls in NIST SP 800-53 and CMMC, published with its model generation, fine-tuning and evaluation code.

  • peekaboolean

    github.com

    A small vision-language model, fine-tuned with LoRA on a 500-million-parameter backbone, that answers typed multiple-choice, ordinal-score and yes/no questions about an image with calibrated probabilities, in about 400 milliseconds on a laptop.