Twin Delayed DDPG

AI and Machine Learning · Reinforcement Learning · 2018 · also: TD3 · td3.yaml

Take the smaller of two critics, update the actor every other step, and smooth the target over nearby actions. Three small changes aimed at one failure: the actor's gradient ascent finds whatever the critic overrates and goes there.

• supersedescorrects · extends
• classifiesspecializes · part-of
• substitutes forapproximates · alternative-to
• depends onrequires · validates

Colour is the family; a dashed line is the second member of it.

This node

• correctsfixes a defect in Deep Deterministic Policy Gradientthe actor is trained to maximise exactly the critic's own errors
• extendsadds capability to Double DQNcarries the two-estimator fix into continuous actions, as a min over the pair

References

• Addressing Function Approximation Error in Actor-Critic Methods — Fujimoto, van Hoof, Meger — 2018 · link