Estimating the Significance of the Added Skill from Initializations: The Case of Decadal Predictions

A considerable part of the skill in decadal forecasts often comes from the forcings, which are present in both initialized and uninitialized model experiments. This makes the added value from initialization difficult to assess. We investigate statistical tests to quantify if initialized forecasts provide skill over the uninitialized experiments. We consider three correlation-based statistics previously used in the literature. The distributions of these statistics under the null hy-pothesis that initialization has no added values are calculated by a surrogate data method. We present some simple exam-ples and study the statistical power of the tests. We find that there can be large differences in both the values and power for the different statistics. In general, the simple statistic defined as the difference between the skill of the initialized and uninitialized experiments behaves best. However, for all statistics the risk of rejecting the true null hypothesis is too high compared to the nominal value. We compare the three tests on initialized decadal predictions (hindcasts) of near-surface temperature performed with a climate model and find evidence for a significant effect of initializations for small lead times. In contrast, we find only little evidence for a significant effect of initializations for lead times longer than 3 years when the experience from the simple experiments is included in the estimation.

Automated summary

This paper investigates statistical tests to quantify the skill difference between initialized and uninitialized forecasts, finding that different statistics methods may overestimate the role of initialization. Based on the long-term prediction of near-surface temperature, it is found that initialization has a significant effect in small lead times but not for lead times longer than 3 years when considering the results from simple experiments.