Experience Replay

AI and Machine Learning · Reinforcement Learning · 2013 · experience-replay.yaml

Stores transitions in a buffer and samples them at random, so an update is not dominated by whatever just happened.

• supersedescorrects · extends
• classifiesspecializes · part-of
• substitutes forapproximates · alternative-to
• depends onrequires · validates

Colour is the family; a dashed line is the second member of it.

This node

• correctsfixes a defect in Deep Q-Networkconsecutive transitions are correlated

Referenced by

• alternative-toAsynchronous Advantage Actor-Critic is a competing approach to thisparallel environments decorrelate the updates instead of a buffer
• requiresOffline RL does not work without thisthe buffer is the whole world, and nothing is ever added to it
• correctsPrioritized Experience Replay fixes a defect in thisuniform sampling spends most updates on transitions with no error left
• requiresSoft Actor-Critic does not work without thisits updates come from the buffer, not from the latest rollout

References

• Playing Atari with Deep Reinforcement Learning — Mnih et al. — 2013 · link