Machine Learning Foundations

L1 Regularization, explained

L1 regularization adds a penalty based on the absolute values of model weights and often encourages sparse solutions.
Back to Glossary

Part of the Semantic Notion technical glossary.

Short Definition

L1 regularization adds a penalty based on the absolute values of model weights and often encourages sparse solutions.

Intuition

L1 pressure can push some weights all the way to zero, making the model ignore less useful inputs.

Technical Definition

L1 regularization adds a term proportional to the sum of absolute parameter values to the training objective.

Example

A linear model with L1 regularization may select a small subset of predictive features from many candidates.

Common Misunderstandings

L1 is not automatically better than L2; the desired behavior is different.
Sparsity depends on the model, data, and regularization strength.

Same Category

More from Machine Learning Foundations

BackpropagationBackpropagation is the algorithm used to compute gradients through a neural network efficiently.Bias-Variance TradeoffThe bias-variance tradeoff describes the balance between models that are too simple to capture structure and models that are too sensitive to training data noise.Binary ClassificationBinary classification is a prediction task where each example belongs to one of two classes.DropoutDropout is a regularization technique that randomly disables parts of a neural network during training.FeatureA feature is an input signal used by a model to make a prediction or learn a representation.Feature EngineeringFeature engineering is the process of transforming raw data into model inputs that make useful patterns easier to learn.

Related Reading

Essays that use this concept

Machine Learning FoundationsL1 vs L2 Regularization vs Dropout

Start here

Need the broader concept map?

Return to the glossary index to move from this definition into adjacent AI, ML, mathematics, and computer science terms.

Browse Glossary