vLLM.hook
A plugin library that lets developers and researchers inspect, analyze, and steer the internal workings of large language models running on the vLLM inference engine, such as their attention and activations.
vLLM.hook is a modular plugin library for vLLM that exposes a model-agnostic system for introspecting and intervening on a model's internals while it runs: dynamic analysis of attention patterns and attention heads, inspection of activations, and custom intervention behaviors such as activation steering and attention control. It provides an extensible worker and analyzer abstraction, making it straightforward to define new hooks, analyzers, and behaviors, with example applications including safety guardrails, reranking, and enhanced instruction following.
The project reports a detailed benchmark comparing vLLM-Hook against vLLM's native hidden-state extraction path: vLLM-Hook's last-token mode gives significantly lower, prompt-length-invariant latency when only the final-position representation is needed, and its all-tokens mode is numerically equivalent to the native path while avoiding the GPU memory overhead of loading a speculative-decoding drafter model. Each use case, such as an attention tracker or a hidden-states extractor, can be configured across execution path, storage backend, disk format, and save mode. The project was presented at ICML 2026 and is installed by cloning the repository and installing its plugin package with pip.