Short Definition
RoPE is a positional encoding method that rotates query and key vectors so attention depends on relative position.
Intuition
Instead of adding a position vector, RoPE changes the geometry of queries and keys according to where tokens appear.
Technical Definition
Rotary Position Embedding applies position-dependent rotations to query and key dimensions, making their dot product encode relative offsets.
Example
Many modern open-weight language models use RoPE to support flexible long-context behavior.
Common Misunderstandings
RoPE is a relative-position method, but it is not the only one.
RoPE alone does not solve every long-context problem.