Dropout

AI and Machine Learning · Training and Optimisation · 2014 · dropout.yaml

Randomly zeroes units during training, so no unit can rely on any particular other one being present. Approximates averaging over an exponential family of thinned networks.

• supersedescorrects · extends
• classifiesspecializes · part-of
• substitutes forapproximates · alternative-to
• depends onrequires · validates

Colour is the family; a dashed line is the second member of it.

This node

• correctsfixes a defect in Backpropagationunits co-adapt and memorise instead of generalising

Referenced by

• alternative-toWeight Decay is a competing approach to thisconstrains weight magnitude rather than co-adaptation

References

• Background — Dropout (neural networks) on Wikipedia