Efficient Hyperparameter Optimization for Convolution Neural Networks in Deep Learning: A Distributed Particle Swarm Optimization Approach

Convolution neural network (CNN) is a kind of powerful and efficient deep learning approach that has obtained great success in many real-world applications. However, due to its complex network structure, the intertwining of hyperparameters, and the time-consuming procedure for network training, finding an efficient network configuration for CNN is a challenging yet tough work. To efficiently solve the hyperparameters setting problem, this paper proposes a distributed particle swarm optimization (DPSO) approach, which can optimize the hyperparameters to find high-performing CNNs. Compared to tedious, historical-experience-based, and personal-preference-based manual designs, the proposed DPSO approach can evolve the hyperparameters automatically and globally to obtain promising CNNs, which provides a new idea and approach for finding the global optimal hyperparameter combination. Moreover, by cooperating with the distributed computing techniques, the DPSO approach can have a considerable speedup when compared with the traditional particle swarm optimization (PSO) algorithm. Extensive experiments on widely-used image classification benchmarks have verified that the proposed DPSO approach can effectively find the CNN model with promising performance, and at the same time, has greatly reduced the computational time when compared with traditional PSO.