Machine Learning Foundations

Dropout, explained

Dropout is a regularization technique that randomly disables parts of a neural network during training.
Back to Glossary

Part of the Semantic Notion technical glossary.

Short Definition

Dropout is a regularization technique that randomly disables parts of a neural network during training.

Intuition

By preventing units from relying too heavily on each other, dropout pushes the network to learn more robust representations.

Technical Definition

Dropout samples a binary mask over activations during training and rescales activations so inference can use the full network.

Example

A feed-forward classifier may drop 20 percent of hidden activations during training to reduce overfitting.

Common Misunderstandings

Dropout is usually disabled during inference.
Dropout is not a replacement for good validation or data hygiene.

Same Category

More from Machine Learning Foundations

BackpropagationBackpropagation is the algorithm used to compute gradients through a neural network efficiently.Bias-Variance TradeoffThe bias-variance tradeoff describes the balance between models that are too simple to capture structure and models that are too sensitive to training data noise.Binary ClassificationBinary classification is a prediction task where each example belongs to one of two classes.FeatureA feature is an input signal used by a model to make a prediction or learn a representation.Feature EngineeringFeature engineering is the process of transforming raw data into model inputs that make useful patterns easier to learn.GeneralizationGeneralization is a model's ability to perform well on new data, not just the data used during training.

Related Reading

Essays that use this concept

Machine Learning FoundationsL1 vs L2 Regularization vs Dropout

Start here

Need the broader concept map?

Return to the glossary index to move from this definition into adjacent AI, ML, mathematics, and computer science terms.

Browse Glossary