4.5 Article

Reward Maximization Through Discrete Active Inference

期刊

NEURAL COMPUTATION
卷 35, 期 5, 页码 807-852

出版社

MIT PRESS
DOI: 10.1162/neco_a_01574

关键词

-

向作者/读者索取更多资源

Active inference is a probabilistic framework based on the principle of minimizing free energy, used for modeling the behavior of biological and artificial agents. It has been successfully applied to various situations involving reward maximization, often yielding comparable or superior results to alternative approaches. This article explores the connection between reward maximization and active inference and demonstrates the conditions under which active inference produces the optimal solution to the Bellman equation, a fundamental equation in reinforcement learning and control. Additionally, it introduces a new recursive active inference scheme that can produce Bellman optimal actions on any finite temporal horizon.
Active inference is a probabilistic framework for modeling the behavior of biological and artificial agents, which derives from the principle of minimizing free energy. In recent years, this framework has been applied successfully to a variety of situations where the goal was to maximize reward, often offering comparable and sometimes superior performance to alternative approaches. In this article, we clarify the connection between reward maximization and active inference by demonstrating how and when active inference agents execute actions that are optimal for maximizing reward. Precisely, we show the conditions under which active inference produces the optimal solution to the Bellman equation, a formulation that underlies several approaches to model-based reinforcement learning and control. On partially observed Markov decision processes, the standard active inference scheme can produce Bellman optimal actions for planning horizons of 1 but not beyond. In contrast, a recently developed recursive active inference scheme (sophisticated inference) can produce Bellman optimal actions on any finite temporal horizon. We append the analysis with a discussion of the broader relationship between active inference and reinforcement learning.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.5
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据