Journal
IEEE INTERNET OF THINGS JOURNAL
Volume 6, Issue 5, Pages 8357-8371Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/JIOT.2019.2917066
Keywords
Autonomous UAV; CNNs; end-to-end learning; nano-UAV; ultralow-power
Categories
Funding
- EC [732631, 780788]
- Swiss National Center of Competence Research Robotics
- SNSF-ERC Starting Grant
Ask authors/readers for more resources
Fully autonomous miniaturized robots (e.g., drones), with artificial intelligence (AI)-based visual navigation capabilities, are extremely challenging drivers of Internet-of-Things edge intelligence capabilities. Visual navigation based on AI approaches, such as deep neural networks (DNNs) are becoming pervasive for standard-size drones, but are considered out of reach for nano-drones with a size of a few cm(2). In this paper, we present the first (to the best of our knowledge) demonstration of a navigation engine for autonomous nano-drones capable of closed-loop end-to-end DNN-based visual navigation. To achieve this goal we developed a complete methodology for parallel execution of complex DNNs directly on board resource-constrained milliwatt-scale nodes. Our system is based on GAP8, a novel parallel ultralow-power computing platform, and a 27-g commercial, open-source Crazyflie 2.0 nano-quadrotor. As part of our general methodology, we discuss the software mapping techniques that enable the DroNet state-of-the-art deep convolutional neural network to be fully executed aboard within a strict 6 frame-per-second real-time constraint with no compromise in terms of flight results, while all processing is done with only 64 mW on average. Our navigation engine is flexible and can be used to span a wide performance range: at its peak performance corner, it achieves 18 frames/s while still consuming on average just 3.5% of the power envelope of the deployed nano-aircraft. To share our key findings with the embedded and robotics communities and foster further developments in autonomous nano-unmanned aerial vehicles (UAVs), we publicly release all our code, datasets, and trained networks.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available