A Benchmark to Test Generalization Capabilities of Deep Learning Methods to Classify Severe Convective Storms in a Changing Climate

This is a test case study assessing the ability of deep learning methods to generalize to a future climate (end of 21st century) when trained to classify thunderstorms in model output representative of the present-day climate. A convolutional neural network (CNN) was trained to classify strongly rotating thunderstorms from a current climate created using the Weather Research and Forecasting model at high-resolution, then evaluated against thunderstorms from a future climate and found to perform with skill and comparatively in both climates. Despite training with labels derived from a threshold value of a severe thunderstorm diagnostic (updraft helicity), which was not used as an input attribute, the CNN learned physical characteristics of organized convection and environments that are not captured by the diagnostic heuristic. Physical features were not prescribed but rather learned from the data, such as the importance of dry air at mid-levels for intense thunderstorm development when low-level moisture is present (i.e., convective available potential energy). Explanation techniques also revealed that thunderstorms classified as strongly rotating are associated with learned rotation signatures. Results show that the creation of synthetic data with ground truth is a viable alternative to human-labeled data and that a CNN is able to generalize a target using learned features that would be difficult to encode due to spatial complexity. Most importantly, results from this study show that deep learning is capable of generalizing to future climate extremes and can exhibit out-of-sample robustness with hyperparameter tuning in certain applications. Plain Language Summary As temperatures and water vapor continue increasing due to climate change, models that were trained using past data may no longer perform with skill. Here, we explored whether the performance of a machine learning model was sensitive to a changing climate. The purpose of the machine learning model was to classify thunderstorms that were created using a high-resolution numerical model into two groups: potentially severe thunderstorms and potentially non-severe thunderstorms. Potentially severe thunderstorms were of interest because they have a greater likelihood of producing tornadoes and large hail, which cause billions of losses and dozens of fatalities every year. Results show that the machine learning model was able to classify thunderstorms with skill in both the present day and future climates partly due to the architecture of the machine learning model. We also explored the reasons behind the machine learning model's skill and found that it was able to learn thunderstorm characteristics and weather information from data. These results provide us with added confidence that machine learning models can learn physical relationships from weather and climate data and perform with skill in a changing climate in certain applications.

Automated summary

This case study investigates the ability of deep learning methods to classify thunderstorms in a future climate based on training data from the present-day climate. The results show that a convolutional neural network performed well in both present and future climates, learning physical characteristics of thunderstorms and environments. This study demonstrates that deep learning can generalize to future climate conditions and exhibit robustness with hyperparameter tuning in certain applications.