MLPs Are Hebbians
Research code that reproduces the experiments behind the paper 'MLPs are Hebbians: Constructing Efficient Fact-Storing MLPs for Transformers', including Hebbian MLP construction, Transformer integration and fact-editing experiments.
This repository holds the code for MLPs are Hebbians: Constructing Efficient Fact-Storing MLPs for Transformers, accepted to COLM 2026. It reproduces the paper's experiments, covering Hebbian MLP constructions, their integration into Transformers, activation experiments on Qwen3 models, and fact-editing experiments.
A single script, scripts/paper/run.py, lists the available experiment targets and can either run an experiment and generate its plot or replot an existing result from saved data. A separate guide groups the results by experiment type and links to their implementations, including further instructions for the Qwen3 experiments.
Installing it needs Python 3.10 or 3.11, with an optional extra that pulls in the Hugging Face dependencies used for the Qwen3 activation experiments.