Target Network

AI and Machine Learning · Reinforcement Learning · 2015 · target-network.yaml

Compute the bootstrap target from a frozen copy of the Q-network, refreshed every few thousand steps. Two lines of code, and the difference between DQN converging and DQN diverging.

• supersedescorrects · extends
• classifiesspecializes · part-of
• substitutes forapproximates · alternative-to
• depends onrequires · validates

Colour is the family; a dashed line is the second member of it.

This node

• correctsfixes a defect in Deep Q-Networka target computed from the weights being updated moves with every step

Referenced by

• requiresDeep Deterministic Policy Gradient does not work without thisthe same bootstrap divergence, now in the actor as well
• requiresDouble DQN does not work without thisthe independent second estimator is the frozen copy already kept

References

• Human-level control through deep reinforcement learning — Mnih et al. — 2015 · link