A Study of Weather-Image Classification Combining VIT and a Dual Enhanced-Attention Module

A weather-image-classification model combining a VIT (vision transformer) and dual augmented attention module is proposed to address the problems of the insufficient feature-extraction capability of traditional deep-learning methods with the recognition accuracy still to be improved and the limited types of weather phenomena existing in the dataset. A pre-trained model vision transformer is used to acquire the basic semantic feature representation of weather images. Dual augmented attention combined with convolutional self-attention and Atrous self-attention modules are used to acquire the low-level and high-level deep-image semantic representations, respectively, and the feature vectors are spliced and fed into the linear layer to obtain the weather types. Experimental validation is performed on the publicly available standard weather-image datasets MWD (Multi-class Weather Database) and WEAPD (Weather Phenomenon Database), and the two datasets are combined to enhance the comprehensiveness of the model for weather-phenomena recognition. The results show that the model achieves the highest F1 scores of 97.47%, 87.69% and 92.73% on the MWD, WEAPD and merged datasets, respectively. These scores are higher than the scores of recent deep-learning models with excellent performance in the experimental comparison, thereby, proving the effectiveness of the model.

Automated summary

A weather-image-classification model is proposed that combines a VIT and dual augmented attention module to overcome the limitations of traditional deeplearning methods in feature extraction, recognition accuracy, and the limited types of weather phenomena in datasets. The model utilizes a pre-trained VIT to acquire basic semantic features and incorporates dual augmented attention with convolutional self-attention and Atrous self-attention modules to capture low-level and high-level deep-image semantics. Experimental validation on standard weather-image datasets demonstrates the model's effectiveness, achieving higher F1 scores than recent deep-learning models in the comparison.