Scoring interval forecasts: Equal-tailed, shortest, and modal interval

We consider different types of predictive intervals and ask whether they are elicitable, that is, are unique minimizers of a loss or scoring function in expectation. The equal-tailed interval is elicitable, with a rich class of suitable loss functions, though subject to translation invariance, or positive homogeneity and differentiability, the Winkler interval score becomes a unique choice. The modal interval also is elicitable, with a sole consistent scoring function, up to equivalence. However, the shortest interval fails to be elicitable relative to practically relevant classes of distributions. These results provide guidance in interval forecast evaluation and support recent choices of performance measures in forecast competitions.

Automated summary

This study examines the elicitable properties of different types of predictive intervals, finding that the equal-tailed interval and the modal interval are elicitable, while the shortest interval is not. These results provide guidance in interval forecast evaluation and support recent choices of performance measures in forecast competitions.