Short Definition
Sparse routing selects only a small subset of model components or experts for each input.
Intuition
Instead of using the whole model every time, the router chooses the parts most relevant to the current token.
Technical Definition
Sparse routing uses learned gating scores, top-k selection, capacity limits, and load-balancing objectives to dispatch examples or tokens to experts.
Example
A top-k router sends each token to a few MoE experts rather than all experts.
Common Misunderstandings
Sparse routing can reduce active compute, but the full model still must be stored or served somehow.
Routing quality and load balance affect model stability.