Short Definition
Cross-entropy measures how well one probability distribution matches another, often a model prediction against a target label.
Intuition
It heavily penalizes confident wrong predictions and rewards assigning high probability to the right answer.
Technical Definition
Cross-entropy is the expected negative log probability assigned by a predicted distribution to outcomes from a target distribution.
Example
A classifier uses cross-entropy loss to train softmax class probabilities.
Common Misunderstandings
Cross-entropy is related to but not identical to entropy.
Low cross-entropy on training data does not guarantee generalization.