Rainbow

AI and Machine Learning · Reinforcement Learning · 2017 · rainbow.yaml

Six DQN improvements in one agent: double, dueling, prioritized replay, multi-step returns, distributional values and noisy exploration, with an ablation for each. The result is mostly the finding that they address different problems and therefore add up.

• supersedescorrects · extends
• classifiesspecializes · part-of
• substitutes forapproximates · alternative-to
• depends onrequires · validates

Colour is the family; a dashed line is the second member of it.

This node

• extendsadds capability to Deep Q-Networkevery fix had been measured alone, against the same unimproved baseline
• requiresdoes not work without Prioritized Experience Replayno single ablation costs more than removing it
• requiresdoes not work without Distributional Reinforcement Learningit pays off only late in training, where shorter papers stopped looking

References

• Rainbow: Combining Improvements in Deep Reinforcement Learning — Hessel et al. — 2017 · link