期刊
ACM JOURNAL ON EMERGING TECHNOLOGIES IN COMPUTING SYSTEMS
卷 13, 期 3, 页码 -出版社
ASSOC COMPUTING MACHINERY
DOI: 10.1145/3005348
关键词
Deep convolutional neural networks; structured pruning; feature map pruning; intra-kernel strided sparsity
类别
资金
- National Research Foundation of Korea (NRF) grant - Korea government (MSIP) [2015R1A2A1A10056051]
- National Research Foundation of Korea [2015R1A2A1A10056051] Funding Source: Korea Institute of Science & Technology Information (KISTI), National Science & Technology Information Service (NTIS)
Real-time application of deep learning algorithms is often hindered by high computational complexity and frequent memory accesses. Network pruning is a promising technique to solve this problem. However, pruning usually results in irregular network connections that not only demand extra representation efforts but also do not fit well on parallel computation. We introduce structured sparsity at various scales for convolutional neural networks: feature map-wise, kernel-wise, and intra-kernel strided sparsity. This structured sparsity is very advantageous for direct computational resource savings on embedded computers, in parallel computing environments, and in hardware-based systems. To decide the importance of network connections and paths, the proposed method uses a particle filtering approach. The importance weight of each particle is assigned by assessing the misclassification rate with a corresponding connectivity pattern. The pruned network is retrained to compensate for the losses due to pruning. While implementing convolutions as matrix products, we particularly show that intra-kernel strided sparsity with a simple constraint can significantly reduce the size of the kernel and feature map tensors. The proposed work shows that when pruning granularities are applied in combination, we can prune the CIFAR-10 network by more than 70% with less than a 1% loss in accuracy.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据