ParamMute
A research framework that improves the faithfulness of retrieval-augmented generation by identifying and suppressing the feed-forward network layers most responsible for a model ignoring retrieved context.
ParamMute is a research framework that investigates why retrieval-augmented language models sometimes ignore the context they were given and answer from their own parametric memory instead. It identifies a subset of mid-to-deep feed-forward network layers that are disproportionately active during these unfaithful answers, then suppresses those layers to calibrate the model toward the retrieved evidence.
The repository provides the training and evaluation pipeline behind the method, including a modified version of the transformers library needed to enable the suppression, and reports results on the CoConflictQA and ConFiQA knowledge-conflict benchmarks. The accompanying paper was accepted at NeurIPS 2025.