Journal
JOURNAL OF MANUFACTURING SYSTEMS
Volume 60, Issue -, Pages 487-499Publisher
ELSEVIER SCI LTD
DOI: 10.1016/j.jmsy.2021.07.015
Keywords
Human-Robot Collaboration; Real-time task scheduling; Multi-agent reinforcement learning
Categories
Funding
- U.S. National Science Foundation (NSF) Grant [CMMI1853454]
Ask authors/readers for more resources
This paper introduces a method of using a chessboard setting to simulate decision-making in HRC assembly processes, optimizing completion time through a Markov game model. The application of a deep-Q-network (DQN) based multi-agent reinforcement learning method is compared with other approaches to improve scheduling efficiency, demonstrating effectiveness in a case study.
Human-Robot Collaboration (HRC) presents an opportunity to improve the efficiency of manufacturing processes. However, the existing task planning approaches for HRC are still limited in many ways, e.g., co-robot encoding must rely on experts' knowledge and the real-time task scheduling is applicable within small stateaction spaces or simplified problem settings. In this paper, the HRC assembly working process is formatted into a novel chessboard setting, in which the selection of chess piece move is used to analogize to the decision making by both humans and robots in the HRC assembly working process. To optimize the completion time, a Markov game model is considered, which takes the task structure and the agent status as the state input and the overall completion time as the reward. Without experts' knowledge, this game model is capable of seeking for correlated equilibrium policy among agents with convergency in making real-time decisions facing a dynamic environment. To improve the efficiency in finding an optimal policy of the task scheduling, a deep-Q-network (DQN) based multi-agent reinforcement learning (MARL) method is applied and compared with the Nash-Q learning, dynamic programming and the DQN-based single-agent reinforcement learning method. A heightadjustable desk assembly is used as a case study to demonstrate the effectiveness of the proposed algorithm with different number of tasks and agents.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available