Machine Learning Foundations

L2 Regularization, explained

L2 regularization adds a penalty based on squared model weights and discourages overly large parameter values.
Back to Glossary

Part of the Semantic Notion technical glossary.

Short Definition

L2 regularization adds a penalty based on squared model weights and discourages overly large parameter values.

Intuition

L2 makes the model spread influence more smoothly instead of relying on extreme weights.

Technical Definition

L2 regularization adds a term proportional to the squared Euclidean norm of parameters to the objective.

Example

Weight decay in neural network training is a common L2-style constraint.

Common Misunderstandings

Weight decay and L2 regularization are closely related but can differ depending on the optimizer.
L2 usually shrinks weights rather than making many of them exactly zero.

Same Category

More from Machine Learning Foundations

BackpropagationBackpropagation is the algorithm used to compute gradients through a neural network efficiently.Bias-Variance TradeoffThe bias-variance tradeoff describes the balance between models that are too simple to capture structure and models that are too sensitive to training data noise.Binary ClassificationBinary classification is a prediction task where each example belongs to one of two classes.DropoutDropout is a regularization technique that randomly disables parts of a neural network during training.FeatureA feature is an input signal used by a model to make a prediction or learn a representation.Feature EngineeringFeature engineering is the process of transforming raw data into model inputs that make useful patterns easier to learn.

Related Reading

Essays that use this concept

Machine Learning FoundationsL1 vs L2 Regularization vs Dropout

Start here

Need the broader concept map?

Return to the glossary index to move from this definition into adjacent AI, ML, mathematics, and computer science terms.

Browse Glossary