Training Algorithm Performance for Image Classification by Neural Networks

Adaptive training is critical for image classification by artificial neural networks (ANNS). While the machine learning community has been enthusiastic in developing various training algorithms, little research has been conducted to evaluate the performance of these algorithms in image classification by neural networks. We introduce and evaluate nine commonly-used training algorithms in terms of their performance in land-cover classification from remotely sensed data by the multi-layer-perceptron (MLP) neural networks. MLP has been considered as the most popular neural network architecture. The training algorithms we consider are Steepest Gradient Descent, Gradient Descent with Momentum, Resilient Propagation, Fletcher-Reeves, Polak-Ribiere, Powell-Beale, Scaled Conjugate Gradient, BFGS (Broyden, Fletcher, Goldfarb, and Shanno), and Levenberg-Marquardt. We use each algorithm to train the MLP networks multiple times using identical training samples, and then apply each of the resultant network models to derive land-cover information from a Landsat Enhanced Thematic Mapper Plus (ETM+) image. The training algorithms are further evaluated according to their training efficiency, capability of convergence, classification accuracy, and stability of the classification accuracy. It is found that the performance of these algorithms varies substantially and selecting an appropriate algorithm can lead to a fast and efficient training and an increase in land-cover classification accuracy by artificial neural networks.