Sessa: Selective State Space Attention
A decoder architecture that folds self-attention into a recurrent feedback pathway, combining input-dependent attention routing with feedback-based recurrent aggregation to improve long-context handling beyond standard Transformers and state-space models such as Mamba.
Sessa is a decoder architecture that integrates self-attention into a recurrent feedback pathway, aiming to preserve and integrate long-context information more effectively than either standard Transformers or state-space models such as Mamba, by combining input-dependent attention routing with feedback-based recurrent aggregation.
It installs by cloning the repository and installing it as an editable Python package, and optionally supports FlashAttention, which is disabled by default and enabled with a use_flash flag; without it, the code falls back to a reference attention implementation.
The repository accompanies a paper describing the architecture, published under the arXiv identifier 2604.18580.