Visibility Maximization Controller for Robotic Manipulation

Occlusions caused by a robot's own body is a common problem for closed-loop control methods employed in eye-to-hand camera setups. We propose an optimization-based reactive controller that minimizes self-occlusions while achieving a desired goal pose. The approach allows coordinated control between the robot's base, arm and head by encoding the line-of-sight visibility to the target as a soft constraint along with other task-related constraints, and solving for feasible joint and base velocities. The generalizability of the approach is demonstrated in simulated and real-world experiments, on robots with fixed or mobile bases, with moving or fixed objects, and multiple objects. The experiments revealed a trade-off between occlusion rates and other task metrics. While a planning-based baseline achieved lower occlusion rates than the proposed controller, it came at the expense of highly inefficient paths and a significant drop in the task success. On the other hand, the proposed controller is shown to improve visibility to the target object(s) without sacrificing too much from the task success and efficiency. Videos and code can be found at: rhys-newbury.github.io/projects/vmc/.

Automated summary

The study introduces an optimization-based reactive controller to minimize occlusions caused by a robot's own body in eye-to-hand camera setups, while achieving a desired goal pose and allowing coordinated control between the robot's base, arm, and head. Demonstrated in simulated and real-world experiments, the approach shows a trade-off between occlusion rates and task metrics, and the proposed controller improves visibility to the target object(s) without sacrificing task success and efficiency too much.