A Survey on Evolutionary Construction of Deep Neural Networks

Automated construction of deep neural networks (DNNs) has become a research hot spot nowadays because DNN's performance is heavily influenced by its architecture and parameters, which are highly task-dependent, but it is notoriously difficult to find the most appropriate DNN in terms of architecture and parameters to best solve a given task. In this work, we provide an insight into the automated DNN construction process by formulating it into a multilevel multiobjective large-scale optimization problem with constraints, where the nonconvex, nondifferentiable, and black-box nature of this problem make evolutionary algorithms (EAs) to stand out as a promising solver. Then, we give a systematical review of existing evolutionary DNN construction techniques from different aspects of this optimization problem and analyze the pros and cons of using EA-based methods in each aspect. This work aims to help DNN researchers to better understand why, where, and how to utilize EAs for automated DNN construction and meanwhile, help EA researchers to better understand the task of automated DNN construction so that they may focus more on EA-favored optimization scenarios to devise more effective techniques.

Automated summary

Automated construction of deep neural networks is a challenging research topic due to the difficulty of finding the most suitable architecture and parameters for a given task. This study formulates the process as a multilevel multiobjective optimization problem and explores the use of evolutionary algorithms for solving it. By reviewing existing techniques, the study aims to provide insights for researchers to effectively utilize EAs in automated DNN construction.