A new equitable score suitable for verifying precipitation in numerical weather prediction

A new equitable score is developed for monitoring precipitation forecasts and for guiding forecast system development. To accommodate the difficult distribution of precipitation, the score measures the error in 'probability space' through use of the climatological cumulative distribution function. For sufficiently skilful forecasting systems, the new score is less sensitive to sampling uncertainty than other established scores. It is therefore called here the 'Stable Equitable Error in Probability Space' (SEEPS). Weather is partitioned into three categories: 'dry', 'light precipitation' and 'heavy precipitation'. SEEPS adapts to the climate of the region in question so that it assesses the salient aspects of the local weather, encouraging 'refinement' and discouraging 'hedging'. To permit continuous monitoring of a system with resolution increasing in time, forecasts are verified against point observations. With some careful choices, observation error and lack of representativeness of model grid-box averages are found to have relatively little impact. SEEPS can identify key forecasting errors including the overprediction of drizzle, failure to predict heavy large-scale precipitation and incorrectly locating convective cells. Area averages are calculated taking into account the observation density. A gain of similar to 2 days, at lead times of 3-9 days, over the last 14 years is found in extratropical scores of forecasts made at the European Centre for Medium-Range Weather Forecasts (ECMWF). This gain is due to system improvements, not the increased amount of data assimilated. SEEPS may also be applicable for verifying other quantities that suffer from difficult spatio-temporal distributions. Copyright (C) 2010 Royal Meteorological Society