Learn Attention
A short visual explanation of how attention works in transformer models, written for students trying to build an intuitive picture of the mechanism.
Visit website (opens in a new tab)lucidrains.github.io
Learn Attention is a small, work-in-progress page that explains how attention appears to its author when reasoning about transformer models, aimed at students and others trying to understand the mechanism intuitively rather than through equations alone. It cites the original 2014 paper on neural machine translation by jointly learning to align and translate, which introduced the attention mechanism the page explains.