TransformerLens
A library for mechanistic interpretability of GPT-style language models that exposes a model's internal activations so they can be cached, edited, removed or replaced while the model runs.
TransformerLens is a library for mechanistic interpretability of GPT-style language models, aiming to help researchers take a trained model and reverse engineer the algorithms it learned during training from its weights. It can load more than 15,000 open-source language models across more than 140 architecture families and exposes their internal activations to the user.
Its TransformerBridge class, the recommended way to load a model, preserves raw Hugging Face weights so logits and activations match Hugging Face's own implementation by default, with a compatibility mode available for numeric parity with the library's older HookedTransformer API, which was removed as a loading path in TransformerLens 4.0. Gated model families such as Llama, Mistral and Gemma require a Hugging Face access token.
It installs as a Python package and is licensed under MIT.