Gradient Descent

AI and Machine Learning · Training and Optimisation · gradient-descent.yaml

Step downhill along the negative gradient. Everything below is a modification of this one idea, made to survive contact with real objectives.

• supersedescorrects · extends
• classifiesspecializes · part-of
• substitutes forapproximates · alternative-to
• depends onrequires · validates

Colour is the family; a dashed line is the second member of it.

Referenced by

• requiresBackpropagation does not work without thisit supplies the gradient the update step consumes
• correctsStochastic Gradient Descent fixes a defect in thisa full-batch gradient costs the entire dataset per step

References

• Background — Gradient descent on Wikipedia