Surface fatigue crack identification in steel box girder of bridges by a deep fusion convolutional neural network based on consumer-grade camera images

This study conducts crack identification from real-world images containing complicated disturbance information (cracks, handwriting scripts, and background) inside steel box girders of bridges. Considering the multilevel and multi-scale features of the input images, a modified fusion convolutional neural network architecture is proposed. As input, 350 raw images are taken with a consumer-grade camera and divided into sub-images with resolution of 64 x 64 pixels (67,200 in total). A regular convolutional neural network structure is employed as baseline to demonstrate the accuracy benefits from the proposed fusion convolutional neural network structure. The confusion matrix is defined for prediction performance evaluation on the test set. A total of six additional entire raw images are used to investigate the robustness and feasibility of the proposed approach. A binary conversion process based on the optimal entropy threshold method is applied and closely followed to identify the crack pixels in the sub-images. The effect of the super-resolution inputs on accuracy is investigated. Results show that the trained modified fusion convolutional neural network can automatically detect the cracks, handwriting, and background from the raw images. The recognition errors of the fusion convolutional neural network in both the training and validation processes are smaller than those of the regular convolutional neural network. The super-resolution process hurts the general identification accuracy.