期刊
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
卷 33, 期 12, 页码 7357-7366出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TNNLS.2021.3084856
关键词
Computer architecture; Adaptive systems; Adaptation models; Complexity theory; Training; Shape; Training data; Adaptive; exemplars; filter pruning; network pruning; structured pruning
类别
资金
- National Science Fund for Distinguished Young Scholars [62025603]
- National Natural Science Foundation of China [U1705262, 62072386, 62072387, 62072389, 62002305, 61772443, 61802324, 61702136]
- Guangdong Basic and Applied Basic Research Foundation [2019B1515120049]
- Fundamental Research Funds for the Central Universities [20720200077, 20720200090, 20720200091]
EPruner is an automatic and efficient pruning approach that simplifies algorithm design by introducing adaptive exemplar filters. It breaks the dependency on training data in determining important filters and allows CPU implementation in seconds.
Popular network pruning algorithms reduce redundant information by optimizing hand-crafted models, and may cause suboptimal performance and long time in selecting filters. We innovatively introduce adaptive exemplar filters to simplify the algorithm design, resulting in an automatic and efficient pruning approach called EPruner. Inspired by the face recognition community, we use a message-passing algorithm Affinity Propagation on the weight matrices to obtain an adaptive number of exemplars, which then act as the preserved filters. EPruner breaks the dependence on the training data in determining the ``important'' filters and allows the CPU implementation in seconds, an order of magnitude faster than GPU-based SOTAs. Moreover, we show that the weights of exemplars provide a better initialization for the fine-tuning. On VGGNet-16, EPruner achieves a 76.34%-FLOPs reduction by removing 88.80% parameters, with 0.06% accuracy improvement on CIFAR-10. In ResNet-152, EPruner achieves a 65.12%-FLOPs reduction by removing 64.18% parameters, with only 0.71% top-5 accuracy loss on ILSVRC-2012. Our code is available at https://github.com/lmbxmu/EPruner.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据