A visual reasoning-based approach for mutual-cognitive human-robot collaboration

Human-robot collaboration (HRC) allows seamless communication and collaboration between humans and robots to fulfil flexible manufacturing tasks in a shared workspace. Nevertheless, existing HRC systems lack an efficient integration of robotic and human cognitions. Empowered by advanced cognitive computing, this paper proposes a visual reasoning-based approach for mutual-cognitive HRC. Firstly, a domain-specific HRC knowl-edge graph is established. Next, the holistic manufacturing scene is perceived by visual sensors as a temporal graph. Then, a collaborative mode with similar instructions can be inferred by graph embedding. Lastly, mutual-cognitive decisions are immersed into the Augmented Reality execution loop for intuitive HRC support. (c) 2022 The Author(s). Published by Elsevier Ltd on behalf of CIRP. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/)

Automated summary

This paper proposes a visual reasoning-based approach for mutual-cognitive human-robot collaboration (HRC), which integrates robotic and human cognitions efficiently. It establishes a domain-specific HRC knowledge graph and perceives the holistic manufacturing scene as a temporal graph using visual sensors. Collaborative modes with similar instructions can be inferred by graph embedding, and mutual-cognitive decisions are implemented in the Augmented Reality execution loop for intuitive HRC support.