Generalized Advantage Estimation

AI and Machine Learning · Reinforcement Learning · 2015 · also: GAE · generalized-advantage-estimation.yaml

Blends TD residuals over many horizons with an exponential weight, giving one knob that trades bias against variance in the advantage estimate.

• supersedescorrects · extends
• classifiesspecializes · part-of
• substitutes forapproximates · alternative-to
• depends onrequires · validates

Colour is the family; a dashed line is the second member of it.

This node

• correctsfixes a defect in Policy Gradientraw returns are too noisy to learn from directly
• requiresdoes not work without Actor-Criticthe value function it leans on is the critic

References

• High-Dimensional Continuous Control Using Generalized Advantage Estimation — Schulman, Moritz, Levine, Jordan, Abbeel — 2015 · link