期刊
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
卷 44, 期 11, 页码 7436-7456出版社
IEEE COMPUTER SOC
DOI: 10.1109/TPAMI.2021.3117837
关键词
Computational modeling; Adaptation models; Computer architecture; Adaptive systems; Routing; Deep learning; Training; Dynamic networks; adaptive inference; efficient inference; convolutional neural networks
资金
- National Science and Technology Major Project of the Ministry of Science and Technology of China [2018AAA0100701]
- National Natural Science Foundation of China [61906106, 62022048]
Dynamic neural networks, which can adapt their structures or parameters to different inputs, have notable advantages in terms of accuracy, computational efficiency, and adaptiveness compared to static models. This survey comprehensively reviews the rapidly developing area of dynamic networks, categorizing them into sample-wise, spatial-wise, and temporal-wise models, and discusses important research problems and future directions.
Dynamic neural network is an emerging research topic in deep learning. Compared to static models which have fixed computational graphs and parameters at the inference stage, dynamic networks can adapt their structures or parameters to different inputs, leading to notable advantages in terms of accuracy, computational efficiency, adaptiveness, etc. In this survey, we comprehensively review this rapidly developing area by dividing dynamic networks into three main categories: 1) sample-wise dynamic models that process each sample with data-dependent architectures or parameters; 2) spatial-wise dynamic networks that conduct adaptive computation with respect to different spatial locations of image data; and 3) temporal-wise dynamic models that perform adaptive inference along the temporal dimension for sequential data such as videos and texts. The important research problems of dynamic networks, e.g., architecture design, decision making scheme, optimization technique and applications, are reviewed systematically. Finally, we discuss the open problems in this field together with interesting future research directions.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据