SAE Lens

A Python library for training and analyzing sparse autoencoders on language models, used for mechanistic interpretability research.

Models

Visit website decoderesearch.github.io

SAELens is a library that helps researchers train sparse autoencoders (SAEs), analyze them for mechanistic interpretability, and generate insights into how language models represent information. Its inference code works with any PyTorch-based model, with deeper integration for TransformerLens through a HookedSAETransformer class, and it can also be used with Hugging Face Transformers or NNsight by extracting activations directly.

The library supports downloading and analyzing pre-trained SAEs as well as training new ones, and it works alongside companion projects such as SAE-Vis for visualizing features and SAEBench for benchmarking. It is written in Python and released under the MIT license.

More in Models

See all 405

mini-AGI
github.com
A continual learning language model that trains from scratch on a single 8 GB VRAM GPU, assembling its own architecture and adapting its size to the hardware and data available rather than shipping as a fixed pretrained model.
- Models
Applied Inference Lab: Compliance Control Embedding Stack
github.com
An open-source contrastive-embedding pipeline that maps technical CVE vulnerability descriptions to formal cybersecurity compliance controls in NIST SP 800-53 and CMMC, published with its model generation, fine-tuning and evaluation code.
- Models
- Security
peekaboolean
github.com
A small vision-language model, fine-tuned with LoRA on a 500-million-parameter backbone, that answers typed multiple-choice, ordinal-score and yes/no questions about an image with calibrated probabilities, in about 400 milliseconds on a laptop.
- Models
- Inference

More in Models

mini-AGI

Applied Inference Lab: Compliance Control Embedding Stack

peekaboolean

Cookies