Deep Learning

Sigmoid, explained

Sigmoid is an S-shaped function that maps real-valued inputs into values between 0 and 1.
Back to Glossary

Part of the Semantic Notion technical glossary.

Short Definition

Sigmoid is an S-shaped function that maps real-valued inputs into values between 0 and 1.

Intuition

It squashes a number into a probability-like range, which is useful for binary decisions but can slow deep networks when saturated.

Technical Definition

The logistic sigmoid computes sigma(x) = 1 / (1 + exp(-x)) and has gradients that become small for large positive or negative inputs.

Example

A binary classifier may use sigmoid to estimate the probability of the positive class.

Common Misunderstandings

Sigmoid outputs are not automatically calibrated probabilities.
Sigmoid is less common than ReLU-like activations in hidden layers of deep networks.

Same Category

More from Deep Learning

Activation FunctionAn activation function is a nonlinear operation applied inside a neural network layer.Batch NormalizationBatch normalization normalizes layer activations using batch statistics to make training more stable.ConvolutionA convolution is an operation that applies a small learned filter across local regions of data.Deep LearningDeep learning is machine learning with neural networks that learn layered representations from data.EmbeddingAn embedding is a learned vector representation of an object such as a word, document, image, user, or product.Latent SpaceLatent space is the internal representation space where a model encodes hidden factors or learned structure.

Related Reading

Essays that use this concept

Data Science and EvaluationCalibration vs Accuracy in Machine Learning

Start here

Need the broader concept map?

Return to the glossary index to move from this definition into adjacent AI, ML, mathematics, and computer science terms.

Browse Glossary