Short Definition
A hyperparameter is a configuration choice that controls training or model structure but is not directly learned as a normal model parameter.
Intuition
Hyperparameters are decisions around the learning process, such as how fast to learn or how much regularization to apply.
Technical Definition
Hyperparameters are external configuration variables set by the practitioner, search process, or training system before or during optimization.
Example
Learning rate, batch size, number of layers, and dropout rate are common hyperparameters.
Common Misunderstandings
Hyperparameters can be tuned automatically, but they are still not ordinary learned weights.
The best hyperparameter setting depends on the data, model, and objective.