LSTM

AI and Machine Learning · Sequence Models · 1997 · also: Long short-term memory · lstm.yaml

Adds a cell state with multiplicative gates controlling what is written, kept and read. The gates give the gradient a route through time that neither vanishes nor explodes.

• supersedescorrects · extends
• classifiesspecializes · part-of
• substitutes forapproximates · alternative-to
• depends onrequires · validates

Colour is the family; a dashed line is the second member of it.

This node

• correctsfixes a defect in Recurrent Networkgradients vanish or explode over long sequences

Referenced by

• correctsTransformer fixes a defect in thisrecurrence forbids parallelism across sequence positions

References

• Background — Long short-term memory on Wikipedia