Machine Learning Foundations

Regularization, explained

Regularization is any technique that constrains learning to reduce overfitting and improve generalization.

Part of the Semantic Notion technical glossary.

Short Definition

Regularization is any technique that constrains learning to reduce overfitting and improve generalization.

Intuition

Regularization keeps the model from using unnecessarily complicated explanations when a simpler pattern is likely to work better.

Technical Definition

Regularization modifies the training objective, architecture, or data process to prefer solutions with desired properties such as small weights, sparsity, or robustness.

Example

L2 regularization penalizes large weights so a model relies less on extreme parameter values.

Common Misunderstandings

Regularization is not only one formula; it is a family of techniques.

Too much regularization can cause underfitting.

Same Category

More from Machine Learning Foundations

BackpropagationBackpropagation is the algorithm used to compute gradients through a neural network efficiently.Bias-Variance TradeoffThe bias-variance tradeoff describes the balance between models that are too simple to capture structure and models that are too sensitive to training data noise.Binary ClassificationBinary classification is a prediction task where each example belongs to one of two classes.DropoutDropout is a regularization technique that randomly disables parts of a neural network during training.FeatureA feature is an input signal used by a model to make a prediction or learn a representation.Feature EngineeringFeature engineering is the process of transforming raw data into model inputs that make useful patterns easier to learn.

Essays that use this concept

Machine Learning FoundationsL1 vs L2 Regularization vs Dropout

Start here

Need the broader concept map?

Return to the glossary index to move from this definition into adjacent AI, ML, mathematics, and computer science terms.

Browse Glossary