Stronger Together: Air-Ground Robotic Collaboration Using Semantics

In this work, we present an end-to-end heterogeneous multi-robot system framework where ground robots are able to localize, plan, and navigate in a semantic map created in real time by a high-altitude quadrotor. The ground robots choose and deconflict their targets independently, without any external intervention. Moreover, they perform cross-view localization by matching their local maps with the overhead map using semantics. The communication backbone is opportunistic and distributed, allowing the entire system to operate with no external infrastructure aside from GPS for the quadrotor. We extensively tested our system by performing different missions on top of our framework over multiple experiments in different environments. Our ground robots travelled over 6 km autonomously with minimal intervention in the real world and over 96 km in simulation without interventions.

Automated summary

In this study, we propose an end-to-end heterogeneous multi-robot system framework that enables ground robots to localize, plan, and navigate in a real-time semantic map created by a high-altitude quadrotor. The ground robots autonomously choose and resolve targets, and perform cross-view localization with the overhead map. The entire system operates with an opportunistic and distributed communication backbone, requiring no external infrastructure aside from GPS for the quadrotor.