Hybrid Uncalibrated Visual Servoing Control of Harvesting Robots With RGB-D Cameras

Visual servoing (VS) control has seen wide adoption in harvesting robots. However, parameter calibration is cumbersome, which makes the use of VS robotic systems inconvenient. Besides, dynamic fruits usually lead to a degeneration of control while tracking. To overcome the drawbacks, we present a new image-based uncalibrated visual servoing (IBUVS) control approach, consisting of a hybrid visual configuration and an adaptive tracking controller, referred to as hybrid-IBUVS. Specifically, our hybrid-IBUVS employs an eye-in-hand camera and a fixed red-green-blue-depth camera to construct a hybrid VS system, basing on multiobject detection and edge-computing technologies. Meanwhile, we also propose adaptive laws to online estimate the uncalibrated parameters of the cameras and robot dynamics. Furthermore, our hybrid-IBUVS uses an adaptive tracking controller to guarantee the harvesting robot to track a predefined trajectory to approach a fruit target. By Lyapunov stability theory, asymptotic convergence of the proposed control scheme is rigorously proven. Experimental results demonstrate the effectiveness of the proposed scheme. All shown results supported the research claims.

Automated summary

This research presents a new image-based uncalibrated visual servoing control approach called hybrid-IBUVS, which combines a hybrid visual configuration and an adaptive tracking controller. The approach utilizes eye-in-hand and fixed red-green-blue-depth cameras and incorporates multiobject detection and edge-computing technologies. Adaptive laws are also proposed to estimate the uncalibrated parameters of the cameras and robot dynamics. The effectiveness of the proposed scheme is demonstrated through experimental results, and the convergence of the control scheme is rigorously proven using Lyapunov stability theory.