Actor-Critic

AI and Machine Learning · Reinforcement Learning · actor-critic.yaml

Learns a policy and a value function together, using the value estimate as the baseline the policy gradient is measured against. Most modern policy methods are some form of this.

• supersedescorrects · extends
• classifiesspecializes · part-of
• substitutes forapproximates · alternative-to
• depends onrequires · validates

Colour is the family; a dashed line is the second member of it.

This node

• extendsadds capability to Policy Gradienta learned baseline instead of sampled returns

Referenced by

• requiresGeneralized Advantage Estimation does not work without thisthe value function it leans on is the critic
• alternative-toGroup Relative Policy Optimization is a competing approach to thisa group-relative baseline instead of a learned one

References

• Background — Actor-critic algorithm on Wikipedia