# TinyEngram

Open research code exploring the Engram memory-injection architecture as an alternative to LoRA for fine-tuning language and vision models, extended from Qwen to Stable Diffusion.

TinyEngram is an open research project exploring the Engram architecture, a large-language-model enhancement that integrates a compact N-gram memory module and a gated retrieval mechanism into key transformer layers to improve phrase-level understanding. Built on top of Qwen, it provides a lightweight, ready-to-train codebase for reproducing, experimenting with, or extending Engram-style models, sharing experiments, training logs, and findings alongside the code.

The project reports that Engram-based memory injection outperforms LoRA in both parameter efficiency and resistance to catastrophic forgetting when fine-tuning language models, and extends the same idea to vision models such as Stable Diffusion for lightweight, composable concept injection, describing this as a modality-agnostic memory architecture rather than one specific to text.

It provides a pinned dependency file and is set up with conda; CUDA notes and optional evaluation dependencies are documented separately.

Website: https://github.com/AutoArk/TinyEngram