TMCrack-Net: A U-Shaped Network with a Feature Pyramid and Transformer for Mural Crack Segmentation

The detection of crack information is very important in mural conservation. In practice, the number of ancient murals is scarce, and the difficulty of collecting digital information about murals leads to minimal data being collected. Crack information appears in pictures of paintings, which resembles painting traces and is easy to misidentify. However, the current mainstream semantic segmentation networks directly use the features of the backbone network for prediction, which do not fully use the features at different scales and ignore the differences between the decoder and encoder features. This paper proposes a new U-shaped convolutional neural network with feature pyramids and a transformer called TMCrack-net. Instead of U-Net's jump-join, an AG-BiFPN network is used, which consists of two modules: a channel cross-fusion (CCT) module with a transformer and a bidirectional feature pyramid network. While fully using the information in different network dimensions, the channel cross-fusion module optimizes the final features of each layer to reduce the confounding effect caused by the fusion of features. For the fusion of multi-scale channel information with decoder features, we designed a fusion module based on a channel attention (called FCA) to guide the fusion of enhanced encoder features with decoder features and reduce the ambiguity between the two feature sets. To demonstrate the effectiveness and generalization of the model, TMCrack-Net was evaluated on the Tang Dynasty tomb chamber mural dataset and Crack500. MIou values of 0.7731 and 0.7944 were achieved, respectively, which are better than those of other advanced crack detection methods. The method yields accurate segmentation performance and is advantageous for mural painting crack segmentation tasks.

Automated summary

This paper introduces a new U-shaped convolutional neural network called TMCrack-Net for crack information detection in mural conservation. The authors propose a new network structure and incorporate feature pyramids and Transformer to optimize feature extraction and fusion, addressing the issues of current mainstream networks in crack detection. Experimental results demonstrate the superior performance of this method in crack detection.