Automatic pixel-level crack segmentation in images using fully convolutional neural network based on residual blocks and pixel local weights

Cracks are significant indicators for the evaluation of the structural health and monitoring process. However, manual crack detection is a time-consuming and challenging task due to large areas, complex structure, and safety risks. Deep learning has emerged as a useful technique to automate the crack detection and identification process. For balanced data, existing deep learning models attempt to segment both crack pixels and non-crack pixels equally. However, due to the highly imbalanced ratio between crack pixels and non-crack pixels, the pixel-wise loss is dominantly guided by the non-crack region and has relatively little influence from the crack region. This leads to the low segmentation accuracy for crack pixels. To address the imbalance problem, this work proposes a local weighting factor with a sensitivity map to remove the network biasness and accurately predict the sensitive pixels. Furthermore, we implement a deep fully convolutional neural network for crack pixel segmentation based on residual blocks with a different number of filters in each convolutional operation that segments the crack pixels and non-crack pixels with unbiased probabilities. For performance evaluation, a new Multi Structure Crack Image (MSCI) dataset is built. By using the MSCI dataset, the proposed method achieved 98.19% crack pixel accuracy and 98.13% non-crack pixel accuracy along with 98.16% average accuracy. In addition, the training time for 10 epochs has dramatically decreased and the experimental results show that the proposed crack segmentation network (CSN) architecture along with local weighting factor and sensitivity map has better crack pixel segmentation accuracy than U-Net and SegNet architectures.

Automated summary

Deep learning has been applied to automated crack detection and identification processes. In the case of data imbalance, introducing a local weighting factor and sensitivity map can improve the accuracy of crack pixel segmentation. Based on the newly established MSCI dataset, the proposed method shows excellent performance in crack pixel and non-crack pixel accuracy.