Journal
IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE
Volume 18, Issue 2, Pages 78-93Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/MCI.2023.3245799
Keywords
Costs; Graphics processing units; Computer architecture; Evolutionary computation; Manuals; Network architecture; Search problems
Categories
Ask authors/readers for more resources
This paper proposes a low-cost neural architecture search method (LoNAS) that addresses the problems of existing approaches by designing a variable-architecture encoding strategy, a training-free proxy, and a three-stage evolutionary algorithm. The experimental results show that LoNAS finds network architectures with competitive performance in test accuracy and the number of parameters, using less search time and fewer computational resources.
Neural architecture search (NAS) has been widely studied to design high-performance network architectures automatically. However, existing approaches require more search time and substantial resource consumption due to their intensive architecture evaluations. Moreover, recently developed NAS algorithms are noncompetitive when combining multiple competing and conflicting objectives, e.g., the test accuracy and the number of parameters. In this paper, a low-cost NAS (LoNAS) method is proposed to address these problems. First, a variable-architecture encoding strategy based on a novel Reg Block is designed to construct high accuracy network architectures with few parameters. Second, a training-free proxy based on the neural tangent kernel (NTK) is proposed to accelerate the search process efficiently. Finally, a three-stage evolutionary algorithm (EA) based on multiple-criteria environmental selection and a set of block-based mutation operators are designed to balance exploration and exploitation better. The experimental results show that LoNAS finds network architectures with competitive performance compared to the state-of-the-art architectures in test accuracy and the number of parameters. Moreover, LoNAS uses less search time and fewer computational resources, consuming only 0.02 GPU Days with one GPU on CIFAR-10 and CIFAR-100. Furthermore, the architectures found by LoNAS on CIFAR-10 and CIFAR-100 exhibit good transferability to ImageNet-16-120, with the test accuracy surpassing that of the state-of-the-art network architectures.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available