Efficient Evolutionary Search of Attention Convolutional Networks via Sampled Training and Node Inheritance

The performance of deep neural networks is heavily dependent on its architecture and various neural architecture search strategies have been developed for automated network architecture design. Recently, evolutionary neural architecture search (EvoNAS) has received increasing attention due to the attractive global optimization capability of evolutionary algorithms. However, EvoNAS suffers from extremely high computational costs because a large number of performance evaluations are usually required in evolutionary optimization, and training deep neural networks is itself computationally very expensive. To address this issue, this article proposes a computationally efficient framework for the evolutionary search of convolutional networks based on a directed acyclic graph, in which parents are randomly sampled and trained on each mini-batch of training data. In addition, a node inheritance strategy is adopted so that the fitness of all offspring individuals can be evaluated without training them. Finally, we encode a channel attention mechanism in the search space to enhance the feature processing capability of the evolved neural networks. We evaluate the proposed algorithm on the widely used datasets, in comparison with 30 state-of-the-art peer algorithms. Our experimental results show that the proposed algorithm is not only computationally much more efficient but also highly competitive in learning performance.

Automated summary

The article proposes a computationally efficient framework for evolutionary search of convolutional networks based on a directed acyclic graph, reducing the computational costs of training deep neural networks by using random sampling of parent nodes and a node inheritance strategy, as well as introducing a channel attention mechanism in the search space to enhance feature processing capability. Experimental results show that the algorithm is competitive in terms of computational efficiency and learning performance.