Engram-PEFT
An unofficial, paper-aligned implementation of DeepSeek's Engram architecture that injects a large, sparsely retrieved conditional memory into a transformer model without increasing inference compute.
Engram-PEFT separates static knowledge storage from dynamic reasoning using a sparse retrieval mechanism, letting a model's factual memory scale up without adding inference FLOPs or disturbing its core reasoning path. It exposes a PEFT-style interface for injecting this memory into any transformer-based model, while also supporting stacked-adapter and full fine-tuning workflows through explicit training-mode settings, and can combine with LoRA for hybrid adaptation.
Training can be driven from Python directly or through a YAML configuration file generated by a command-line template, covering the base model, Engram-specific hyperparameters, optional LoRA settings, standard Transformers training arguments and dataset settings. It detects Huawei Ascend NPU hardware automatically and switches its mixed-precision settings accordingly, since NPUs generally do not support bfloat16, and supports distributed training across both NPU and CUDA backends.