Full-body pose estimation for excavators based on data fusion of multiple onboard sensors

To reduce machine-related accidents on sites, automatically monitoring the full-body poses of operating heavy machines is crucial. Conventional pose estimation systems relying on homogeneous sensors are vulnerable to negative environmental impacts, leading to inaccurate and unstable estimation of machine states. Hence, a full -body pose estimation framework is proposed for excavators, with a data fusion strategy to utilize different types of onboard sensors for enhanced accuracy and robustness. Specifically, a non-invasive onboard visual-inertial sensor system is designed for data fusion. Then, through competitive and complementary data fusion, the key -points describing the full-body poses of the excavator are tracked in 3D space. Especially, an EKF-based local-ization algorithm is developed for optimized multi-keypoint tracking, which is verified to improve the accuracy and robustness of pose estimation by a real-world excavator case study. The proposed sensor-fusion method can effectively improve operational safety, by accurately monitoring the motion of heavy machines operating on construction sites.

Automated summary

To reduce accidents involving heavy machines on construction sites, it is crucial to automatically monitor the full-body poses of the operating machines. Conventional pose estimation systems using homogeneous sensors are vulnerable to negative environmental impacts, resulting in inaccurate and unstable estimation of machine states. Hence, a full-body pose estimation framework is proposed for excavators, utilizing a data fusion strategy that incorporates different types of onboard sensors to enhance accuracy and robustness. Through competitive and complementary data fusion, key-points describing the full-body poses of the excavator are tracked in 3D space, improving the accuracy and robustness of pose estimation. Especially, an EKF-based localization algorithm is developed for optimized multi-keypoint tracking and validated through a real-world excavator case study. The proposed sensor-fusion method effectively enhances operational safety by accurately monitoring the motion of heavy machines on construction sites.