Crack Detection and Segmentation Using Deep Learning with 3D Reality Mesh Model for Quantitative Assessment and Integrated Visualization

Crack detection has been an active research topic for civil infrastructure inspection. Over the last few years, many research efforts have focused on applying deep learning-based techniques to automatically detect cracks in images. Good results have been reported with bounding boxes around the detected cracks in images. However, there is no accurate crack segmentation, quantitative assessment, or integrated visualization in the context of engineering structures. In addition, most previously developed deep learning-based crack detection models have been trained with homogenous images collected under controlled conditions, rather than applying the models to images collected during real-world infrastructure inspections. In this paper, two deep learning-based approaches are developed for crack detection and segmentation. The first approach is to integrate the faster region-based convolutional neural network (FRCNN) with structured random forest edge detection (SRFED). The FRCNN is used to detect cracks with bounding boxes while SRFED is applied to segment the cracks within the boxes. The second approach is to directly apply Mask RCNN for crack detection and segmentation. The models have been trained with diverse images collected during real-world infrastructure inspections, enhancing the broad applicability of the models. Both approaches have been applied in a unified framework using three-dimensional (3D) reality mesh-modeling technology that enables quantitative assessment with the integrated visualization of an inspected structure. The effectiveness and robustness of the developed techniques are evaluated and demonstrated using various real cases including bridges, road pavements, underground tunnels, water towers, and buildings.