Journal
IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION
Volume 25, Issue 5, Pages 894-912Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TEVC.2021.3079985
Keywords
Optimization; Computer architecture; Task analysis; Data models; Mathematical model; Computational modeling; Search problems; Automated design of DNNs; deep neural networks; evolutionary algorithms; optimization
Funding
- National Key Research and Development Project, Ministry of Science and Technology, China [2018AAA0101301]
- National Natural Science Foundation of China [61876162]
- Research Grants Council of the Hong Kong SAR [PolyU11202418, PolyU11209219]
- Australian Research Council (ARC) [LP180100114, DP200102611]
Ask authors/readers for more resources
Automated construction of deep neural networks is a challenging research topic due to the difficulty of finding the most suitable architecture and parameters for a given task. This study formulates the process as a multilevel multiobjective optimization problem and explores the use of evolutionary algorithms for solving it. By reviewing existing techniques, the study aims to provide insights for researchers to effectively utilize EAs in automated DNN construction.
Automated construction of deep neural networks (DNNs) has become a research hot spot nowadays because DNN's performance is heavily influenced by its architecture and parameters, which are highly task-dependent, but it is notoriously difficult to find the most appropriate DNN in terms of architecture and parameters to best solve a given task. In this work, we provide an insight into the automated DNN construction process by formulating it into a multilevel multiobjective large-scale optimization problem with constraints, where the nonconvex, nondifferentiable, and black-box nature of this problem make evolutionary algorithms (EAs) to stand out as a promising solver. Then, we give a systematical review of existing evolutionary DNN construction techniques from different aspects of this optimization problem and analyze the pros and cons of using EA-based methods in each aspect. This work aims to help DNN researchers to better understand why, where, and how to utilize EAs for automated DNN construction and meanwhile, help EA researchers to better understand the task of automated DNN construction so that they may focus more on EA-favored optimization scenarios to devise more effective techniques.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available