What is dopamine doing in model-based reinforcement learning?

Experiments have implicated dopamine in model-based reinforcement learning (RL). These findings are unexpected as dopamine is thought to encode a reward prediction error (RPE), which is the key teaching signal in model-free RL. Here we examine two possible accounts for dopamine?s involvement in model-based RL: the first that dopamine neurons carry a prediction error used to update a type of predictive state representation called a successor representation, the second that two well established aspects of dopaminergic activity, RPEs and surprise signals, can together explain dopamine?s involvement in model-based RL.

Automated summary

The involvement of dopamine in model-based reinforcement learning could be explained by its role in carrying prediction errors to update successor representations, or by the combination of well-established aspects of dopaminergic activity, reward prediction errors, and surprise signals.