Journal
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021
Volume -, Issue -, Pages 5016-5026Publisher
IEEE COMPUTER SOC
DOI: 10.1109/CVPR46437.2021.00498
Keywords
-
Funding
- National Natural Science Foundation of China [61876007]
- Australian Research Council [DE180101438, DP210101859]
Ask authors/readers for more resources
The proposed approach in this paper dynamically removes redundant filters by embedding the manifold information of all instances into the space of pruned networks. The effectiveness of the method is verified on several benchmarks, showing better performance in terms of both accuracy and computational cost compared to state-of-the-art methods. This new paradigm maximally excavates redundancy in the network architecture.
Neural network pruning is an essential approach for reducing the computational complexity of deep models so that they can be well deployed on resource-limited devices. Compared with conventional methods, the recently developed dynamic pruning methods determine redundant filters variant to each input instance which achieves higher acceleration. Most of the existing methods discover effective subnetworks for each instance independently and do not utilize the relationship between different inputs. To maximally excavate redundancy in the given network architecture, this paper proposes a new paradigm that dynamically removes redundant filters by embedding the manifold information of all instances into the space of pruned networks (dubbed as ManiDP). We first investigate the recognition complexity and feature similarity between images in the training set. Then, the manifold relationship between instances and the pruned sub-networks will be aligned in the training procedure. The effectiveness of the proposed method is verified on several benchmarks, which shows better performance in terms of both accuracy and computational cost compared to the state-of-the-art methods. For example, our method can reduce 55.3% FLOPs of ResNet-34 with only 0.57% top-I accuracy degradation on ImageNet. The code will be available at https://github.com/huawei-noah/Pruning/tree/master/ManiDP.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available