Markov Decision Process

AI and Machine Learning · Reinforcement Learning · also: MDP · markov-decision-process.yaml

States, actions, transition probabilities and rewards, where the next state depends only on the present one. The formalism every method below is defined against.

• supersedescorrects · extends
• classifiesspecializes · part-of
• substitutes forapproximates · alternative-to
• depends onrequires · validates

Colour is the family; a dashed line is the second member of it.

Referenced by

• part-ofBellman Equation is a subtopic of this
• extendsMaximum Entropy RL adds capability to thisreward alone commits to one of several equally good actions and stops looking
• requiresMonte Carlo Tree Search does not work without thisit searches the tree those transitions define
• specializesMulti-Armed Bandit is a specific case of thisa single state, so no action changes what is faced next
• extendsPotential-Based Reward Shaping adds capability to thisdense guidance can be added to a sparse reward without moving its optimum
• specializesTemporal Difference Learning is a specific case of thisestimates the value function the MDP defines

References

• Background — Markov decision process on Wikipedia