An integrated vision-based system for efficient robot arm teleoperation

Purpose This paper aims to present a natural human-robot teleoperation system, which capitalizes on the latest advancements of monocular human pose estimation to simplify scenario requirements on heterogeneous robot arm teleoperation. Design/methodology/approach Several optimizations in the joint extraction process are carried on to better balance the performance of the pose estimation network. To bridge the gap between human joint pose in Cartesian space and heterogeneous robot joint angle pose in Radian space, a routinized mapping procedure is proposed. Findings The effectiveness of the developed methods on joint extraction is verified via qualitative and quantitative experiments. The teleoperation experiments on different robots validate the feasibility of the system controlling. Originality/value The proposed system provides an intuitive and efficient human-robot teleoperation method with low-cost devices. It also enhances the controllability and flexibility of robot arms by releasing human operator from motion constraints, paving a new way for effective robot teleoperation.

Automated summary

This study presents a natural human-robot teleoperation system utilizing monocular human pose estimation, which simplifies heterogeneous robot arm teleoperation, and bridges the gap between human joint pose and robot joint angle pose. Qualitative and quantitative experiments verify the effectiveness of joint extraction methods, while teleoperation experiments on different robots validate system control feasibility, providing an intuitive and efficient method for human-robot teleoperation with low-cost devices.