Short Definition
Gradient descent is an optimization method that updates model parameters in the direction that most reduces the loss locally.
Intuition
Imagine walking downhill on the loss surface. The gradient tells you which direction is uphill, so gradient descent steps the other way.
Technical Definition
Gradient descent iteratively updates parameters by subtracting a learning-rate-scaled gradient of the objective with respect to those parameters.
Example
A neural network uses gradients from backpropagation to reduce prediction error over many batches.
Common Misunderstandings
Gradient descent is local; it does not guarantee the globally best solution.
Bigger steps are not always faster because they can overshoot good regions.