Journal
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
Volume 60, Issue -, Pages -Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TGRS.2022.3161499
Keywords
Marine vehicles; Field programmable gate arrays; Convolutional neural networks; Adaptation models; Radar polarimetry; Synthetic aperture radar; Feature extraction; Algorithm; hardware codesign; convolutional neural network~(CNN) acceleration; hardware-guided pruning; ship detection; synthetic aperture radar~(SAR) imagery
Categories
Funding
- National Natural Science Foundation of China [62071360, 6207010444, 61801359, 61571345, 91538101, 61501346, 61502367]
- Young Talent fund of University Association for Science and Technology in Shaanxi of China [20190103]
- China Postdoctoral Science Foundation [2017M620440]
- 111 Project [B08038]
- Fundamental Research Funds for the Central Universities [JB180104]
- Natural Science Basic Research Plan in Shaanxi Province of China [2019JQ153, 2016JQ6023, 2016JQ6018]
- Yangtse Rive Scholar Bonus Schemes [CJT160102]
- Ten Thousand Talent Program
Ask authors/readers for more resources
In this article, the authors propose OSCAR-RT, an algorithm/hardware codesign framework for real-time on-satellite CNN-based SAR ship detection. OSCAR-RT simultaneously produces an accurate and hardware-friendly CNN model and an ultra-efficient FPGA-based hardware accelerator. Experimental results demonstrate the effectiveness of OSCAR-RT in achieving high detection accuracy and speed for on-satellite ship detection.
Recently, the convolutional neural network (CNN)-based approach for on-satellite ship detection in synthetic aperture radar (SAR) images has received increasing attention since it does not rely on predefined imagery features and distributions that are required in conventional detection methods. To achieve high detection accuracy, most of the existing CNN-based methods leverage complex off-the-shelf CNN models for optical imagery. Unfortunately, this usually leads to expensive computational cost, which is hard to process in real time using resource-constrained devices deployed in the harsh satellite environment. In this article, we propose OSCAR-RT, the first end-to-end algorithm/hardware codesign framework for real-time on-satellite CNN-based SAR ship detection, which can simultaneously produce an accurate and hardware-friendly CNN model and an ultraefficient field-programmable gate array (FPGA)-based hardware accelerator that can be deployed on satellites. With the real-time on-satellite processing speed in mind, we start from a state-of-the-art compact CNN model for optical imagery. To eliminate the sharp decrease in the detection accuracy for SAR imagery, we analyze the discrepancy between the SAR domain and optical domain and propose to adapt the model by adjusting the output feature size to better detect relatively smaller objects in SAR imagery. To improve the detection speed, we propose to develop a fully pipelined interlayer streaming accelerator architecture, where all the layers of the CNN model can be concurrently processed using on-chip FPGA resources. To achieve this architecture, we first propose a hardware-guided, progressive, and structural pruning strategy, which is guided by our modeled hardware metrics and applies state-of-the-art coarse-grained and fine-grained filter pruning as well as mixed-precision quantization techniques. Moreover, to improve the reusability and portability of the hardware accelerator design, we develop a library of highly optimized CNN components in high-level synthesis, together with their performance and resource models. Finally, we map the pruned CNN model onto these hardware library components in a fully pipelined interlayer streaming fashion, by adjusting their parallelism factors to balance the execution of each layer and fit into the resource constraint. Experimental results using the adapted MobileNetV1, MobileNetV2, and SqueezeNet models on the widely used SAR ship detection dataset (SSDD) demonstrate the effectiveness of OSCAR-RT; for the MobileNetV1 model, it achieves an average precision of 94%, a detection speed of 652 frames/s on the Xilinx VC709 FPGA evaluation board while consuming about 5.8-W power.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available