4.7 Article

Transforming Large-Size to Lightweight Deep Neural Networks for IoT Applications

Journal

ACM COMPUTING SURVEYS
Volume 55, Issue 11, Pages -

Publisher

ASSOC COMPUTING MACHINERY
DOI: 10.1145/3570955

Keywords

Compression; knowledge distillation; pruning; sparse representation

Ask authors/readers for more resources

Deep Neural Networks (DNNs) are popular for their high performance and automated feature extraction capability. However, their deployment on resource-constrained IoT devices is challenging due to the requirements of computation, energy, and storage. Various compression techniques have been proposed to reduce the energy, storage, and computation requirements of DNNs with minimal accuracy compromise. This article provides a comprehensive overview of existing literature on DNN compression techniques and discusses their challenges and applications in IoT.
Deep Neural Networks (DNNs) have gained unprecedented popularity due to their high-order performance and automated feature extraction capability. This has encouraged researchers to incorporate DNN in different Internet of Things (IoT) applications in recent years. However, the colossal requirement of computation, energy, and storage of DNNs make their deployment prohibitive on resource-constrained IoT devices. Therefore, several compression techniques have been proposed in recent years to reduce the energy, storage, and computation requirements of the DNN. These techniques have utilized a different perspective for compressing a DNN with minimal accuracy compromise. This encourages us to comprehensively overview DNN compression techniques for the IoT. This article presents a comprehensive overview of existing literature on compressing the DNN that reduces energy consumption, storage, and computation requirements for IoT applications. We divide the existing approaches into five broad categories-network pruning, sparse representation, bits precision, knowledge distillation, and miscellaneous-based upon the mechanism incorporated for compressing the DNN. The article discusses the challenges associated with each category of DNN compression techniques and presents some prominent applications using IoT in conjunction with a compressed DNN. Finally, we provide a quick summary of existing work under each category with the future direction in DNN compression.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available