Short Definition
Batch normalization normalizes layer activations using batch statistics to make training more stable.
Intuition
It keeps intermediate values in a friendlier range so optimization has an easier job.
Technical Definition
Batch normalization standardizes activations per mini-batch and applies learned scale and shift parameters.
Example
A convolutional network may use batch normalization after convolution and before ReLU.
Common Misunderstandings
Batch normalization behaves differently during training and inference.
It is not always ideal for very small batch sizes or all architectures.