Recurrent Network

AI and Machine Learning · Sequence Models · also: RNN · recurrent-network.yaml

Carry a hidden state forward one step at a time. The natural shape for sequences, and the source of every problem the rest of this cluster fixes.

Colour is the family; a dashed line is the second member of it.

Drag to pan · scroll to zoom · click a node to open it
G attention Attention recurrent-network Recurrent Network attention->recurrent-network a fixed-size hidden state cannot carry a whole sequence gradient-clipping Gradient Clipping gradient-clipping->recurrent-network exploding gradients destroy the weights in one step lstm LSTM lstm->recurrent-network gradients vanish or explode over long sequences

Referenced by

References