Reliable identification of redundant kernels for convolutional neural network compression

To compress deep convolutional neural networks (CNNs) with large memory footprint and long inference time, this paper proposes a novel pruning criterion based on layer-wise L-n-norms of feature maps to identify unimportant convolutional kernels. We calculate the L-n-norm of the feature map outputted by each convolutional kernel to evaluate the importance of the kernel. Furthermore, we use different L-n-norms for different layers, e.g., L-1-norm for the first convolutional layer, L-2-norm for middle convolutional layers and L-infinity-norm for the last convolutional layer. With the ability of accurately identifying unimportant convolutional kernels in each layer, the proposed method achieves a good balance between model size and inference accuracy. Experimental results on CIFAR, SVHN and ImageNet datasets and an application example in a railway intelligent surveillance system show that the proposed method outperforms existing kernel-norm-based methods and is generally applicable to any deep neural network with convolutional operations. (C) 2019 Elsevier Inc. All rights reserved.