Conservative Q-Learning

AI and Machine Learning · Reinforcement Learning · 2020 · also: CQL · conservative-q-learning.yaml

Adds a term that pushes down the value of actions the dataset does not contain, so what comes out is a lower bound on the policy's true value rather than an optimistic guess.

• supersedescorrects · extends
• classifiesspecializes · part-of
• substitutes forapproximates · alternative-to
• depends onrequires · validates

Colour is the family; a dashed line is the second member of it.

This node

• correctsfixes a defect in Offline RLthe critic overrates actions never seen, and the policy walks straight to them

References

• Conservative Q-Learning for Offline Reinforcement Learning — Kumar, Zhou, Tucker, Levine — 2020 · link