Journal
FRONTIERS IN NEUROROBOTICS
Volume 14, Issue -, Pages -Publisher
FRONTIERS MEDIA SA
DOI: 10.3389/fnbot.2020.610139
Keywords
human– robot interaction; intention estimation; scene understanding; visual-NLP; semantics
Funding
- Shenzhen Science and Technology Innovation Commission [JCYJ20170410172100520, JCYJ20170818104502599]
- Shenzhen Institute of Artificial Intelligence and Robotics for Society [2019-INT020]
Ask authors/readers for more resources
This research proposes a novel task-based framework that enables robots to understand human intentions using visual semantics information and satisfy human intentions based on natural language instructions, further advancing human-robot interaction.
With the rapid development of robotic and AI technology in recent years, human-robot interaction has made great advancement, making practical social impact. Verbal commands are one of the most direct and frequently used means for human-robot interaction. Currently, such technology can enable robots to execute pre-defined tasks based on simple and direct and explicit language instructions, e.g., certain keywords must be used and detected. However, that is not the natural way for human to communicate. In this paper, we propose a novel task-based framework to enable the robot to comprehend human intentions using visual semantics information, such that the robot is able to satisfy human intentions based on natural language instructions (total three types, namely clear, vague, and feeling, are defined and tested). The proposed framework includes a language semantics module to extract the keywords despite the explicitly of the command instruction, a visual object recognition module to identify the objects in front of the robot, and a similarity computation algorithm to infer the intention based on the given task. The task is then translated into the commands for the robot accordingly. Experiments are performed and validated on a humanoid robot with a defined task: to pick the desired item out of multiple objects on the table, and hand over to one desired user out of multiple human participants. The results show that our algorithm can interact with different types of instructions, even with unseen sentence structures.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available