Short Definition
A transformer is a neural network architecture built around attention mechanisms and feed-forward layers.
Intuition
Transformers process many positions in parallel while letting tokens exchange information through attention.
Technical Definition
The transformer architecture stacks attention, normalization, residual connections, and position-wise feed-forward networks to model sequences and other tokenized inputs.
Example
Modern language models use decoder-only transformers to predict the next token.
Common Misunderstandings
A transformer is not synonymous with an LLM, though most modern LLMs use transformers.
Transformers still need positional information because attention alone has no built-in order.