Ultra-lightweight CNN design based on neural architecture search and knowledge distillation: A novel method to build the automatic recognition model of space target ISAR images

In this paper, a novel method of ultra-lightweight convolution neural network (CNN) design based on neural architecture search (NAS) and knowledge distillation (KD) is proposed. It can realize the automatic construction of the space target inverse synthetic aperture radar (ISAR) image recognition model with ultra-lightweight and high accuracy. This method introduces the NAS method into the radar image recognition for the first time, which solves the time-consuming and labor-consuming problems in the artificial design of the space target ISAR image automatic recognition model (STIIARM). On this basis, the NAS model's knowledge is transferred to the student model with lower computational complexity by the flow of the solution procedure (FSP) distillation method. Thus, the decline of recognition accuracy caused by the direct compression of model structural parameters can be effectively avoided, and the ultra lightweight STIIARM can be obtained. In the method, the Inverted Linear Bottleneck (ILB) and Inverted Residual Block (IRB) are firstly taken as each block's basic structure in CNN. And the expansion ratio, output filter size, number of IRBs, and convolution kernel size are set as the search parameters to construct a hierarchical decomposition search space. Then, the recognition accuracy and computational complexity are taken as the objective function and constraint conditions, respectively, and the global optimization model of the CNN architecture search is established. Next, the simulated annealing (SA) algorithm is used as the search strategy to search out the lightweight and high accuracy STIIARM directly. After that, based on the three principles of similar block structure, the same corresponding channel number, and the minimum computational complexity, the more lightweight student model is designed, and the FSP matrix pairing between the NAS model and student model is completed. Finally, by minimizing the loss between the FSP matrix pairs of the NAS model and student model, the student model's weight adjustment is completed. Thus the ultra-lightweight and high accuracy STIIARM is obtained. The proposed method's effectiveness is verified by the simulation experiments on the ISAR image dataset of five types of space targets.(c) 2021 China Ordnance Society. Publishing services by Elsevier B.V. on behalf of KeAi Communications Co. Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/ licenses/by-nc-nd/4.0/).

Automated summary

In this paper, a novel method of ultra-lightweight convolution neural network (CNN) design based on neural architecture search (NAS) and knowledge distillation (KD) is proposed. It can automatically construct the space target ISAR image recognition model with ultra-lightweight and high accuracy. The effectiveness of the proposed method is verified by simulation experiments on the ISAR image dataset of five types of space targets.