Residual Connection

AI and Machine Learning · Training and Optimisation · 2015 · also: ResNet, Skip connection · residual-connection.yaml

Add a layer's input to its output so the layer only has to learn a correction. Gradients then reach early layers along a path that does not attenuate, and depth stops being a liability.

• supersedescorrects · extends
• classifiesspecializes · part-of
• substitutes forapproximates · alternative-to
• depends onrequires · validates

Colour is the family; a dashed line is the second member of it.

This node

• correctsfixes a defect in Backpropagationgradients vanish through a deep stack of layers

References

• Deep Residual Learning for Image Recognition — He, Zhang, Ren, Sun — 2015 · link
• Background — Residual neural network on Wikipedia