期刊
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
卷 24, 期 12, 页码 2038-2050出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TNNLS.2013.2271454
关键词
Adaptive dynamic programming; goal representation heuristic dynamic programming; maze navigation/path planning; Markov decision process; reinforcement learning
类别
资金
- National Science Foundation [CAREER ECCS 1053717]
- Army Research Office [W911NF-12-1-0378]
- NSF-DFG [CNS 1117314]
- National Natural Science Foundation of China [51228701, 61075072]
- Program for New Century Excellent Talents in University [NCET-10-0901]
- Direct For Computer & Info Scie & Enginr
- Division Of Computer and Network Systems [1117314] Funding Source: National Science Foundation
- Directorate For Engineering
- Div Of Electrical, Commun & Cyber Sys [1053717] Funding Source: National Science Foundation
Goal representation heuristic dynamic programming (GrHDP) is proposed in this paper to demonstrate online learning in the Markov decision process. In addition to the (external) reinforcement signal in literature, we develop an adaptively internal goal/reward representation for the agent with the proposed goal network. Specifically, we keep the actor-critic design in heuristic dynamic programming (HDP) and include a goal network to represent the internal goal signal, to further help the value function approximation. We evaluate our proposed GrHDP algorithm on two 2-D maze navigation problems, and later on one 3-D maze navigation problem. Compared to the traditional HDP approach, the learning performance of the agent is improved with our proposed GrHDP approach. In addition, we also include the learning performance with two other reinforcement learning algorithms, namely Sarsa(lambda) and Q-learning, on the same benchmarks for comparison. Furthermore, in order to demonstrate the theoretical guarantee of our proposed method, we provide the characteristics analysis toward the convergence of weights in neural networks in our GrHDP approach.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据