Hybrid multimodal fusion with deep learning for rolling bearing fault diagnosis

For vibration signal of rolling bearing with long time series obtained from multiple sampling points, hybrid multimodal fusion with deep learning is proposed for fault diagnosis. Feature-level multimodal fusion method is used to extract time domain features from vibration signal samples of the whole life cycle. Moreover, those features are transformed into multimodal samples, which are composed of grayscale images and time series. Convolutional neural network (CNN), which is commonly applied in image processing, is used to deal with grayscale images, while deep belief network (DBN) is utilized to train time series samples. Subsequently, decision-level multimodal fusion can be achieved by combining several different deep learning models, so as to obtain comprehensive fault prediction result. The effectiveness of the proposed method is verified by rolling bearing datasets with multiple typical faults. Compared with individual deep learning models and other traditional models, the proposed method can achieve higher fault diagnosis accuracy.

Automated summary

This method extracts time-domain features of vibration signals from rolling bearings and uses convolutional neural networks and deep belief networks to process grayscale images and time series samples respectively. By combining multiple deep learning models, comprehensive fault prediction results are obtained, demonstrating higher fault diagnosis accuracy compared to individual deep learning models and traditional methods.