Journal
IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION
Volume 27, Issue 1, Pages 26-36Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TEVC.2022.3227562
Keywords
Convolutional neural network; evolutionary algorithm (EA); neuroevolution; performance predictor
Ask authors/readers for more resources
Neural architecture search (NAS) automates architecture engineering in neural networks, but evaluating candidate networks is computationally expensive. To reduce this overhead, a predictor-assisted evolutionary NAS (PRE-NAS) strategy is proposed, which can perform well even with a small number of evaluated architectures. PRE-NAS leverages evolutionary search strategies and weight inheritance over generations to improve accuracy of predictions. Experimental results show that PRE-NAS outperforms state-of-the-art NAS methods, finding competitive architectures with low test error rates on CIFAR-10 and ImageNet.
Neural architecture search (NAS) aims to automate architecture engineering in neural networks. This often requires a high computational overhead to evaluate a number of candidate networks from the set of all possible networks in the search space. Prediction of the performance of a network can alleviate this high computational overhead by mitigating the need for evaluating every candidate network. Developing such a predictor typically requires a large number of evaluated architectures which may be difficult to obtain. We address this challenge by proposing a novel evolutionary-based NAS strategy, predictor-assisted evolutionary NAS (PRE-NAS) which can perform well even with an extremely small number of evaluated architectures. PRE-NAS leverages new evolutionary search strategies and integrates high-fidelity weight inheritance over generations. Unlike one-shot strategies, which may suffer from bias in the evaluation due to weight sharing, offspring candidates in PRE-NAS are topologically homogeneous. This circumvents bias and leads to more accurate predictions. Extensive experiments on the NAS-Bench-201 and DARTS search spaces show that PRE-NAS can outperform state-of-the-art NAS methods. With only a single GPU searching for 0.6 days, a competitive architecture can be found by PRE-NAS which achieves 2.40% and 24% test error rates on CIFAR-10 and ImageNet, respectively.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available