Machine Learning Foundations

Overfitting, explained

Overfitting happens when a model learns training-set details that do not generalize to new data.
Back to Glossary

Part of the Semantic Notion technical glossary.

Short Definition

Overfitting happens when a model learns training-set details that do not generalize to new data.

Intuition

The model memorizes quirks of the examples instead of learning the pattern that will hold later.

Technical Definition

Overfitting is a generalization failure where training error decreases while validation or test performance worsens.

Example

A decision tree that memorizes every training row may perform poorly on future customers.

Common Misunderstandings

Low training error can be a warning sign, not proof of success.
Overfitting can happen in small and large models.

Same Category

More from Machine Learning Foundations

BackpropagationBackpropagation is the algorithm used to compute gradients through a neural network efficiently.Bias-Variance TradeoffThe bias-variance tradeoff describes the balance between models that are too simple to capture structure and models that are too sensitive to training data noise.Binary ClassificationBinary classification is a prediction task where each example belongs to one of two classes.DropoutDropout is a regularization technique that randomly disables parts of a neural network during training.FeatureA feature is an input signal used by a model to make a prediction or learn a representation.Feature EngineeringFeature engineering is the process of transforming raw data into model inputs that make useful patterns easier to learn.

Related Reading

Essays that use this concept

Data Science and EvaluationOverfitting vs Underfitting vs Generalization

Start here

Need the broader concept map?

Return to the glossary index to move from this definition into adjacent AI, ML, mathematics, and computer science terms.

Browse Glossary