Forecasting in the Presence of Instabilities: How We Know Whether Models Predict Well and How to Improve Them

This article provides guidance on how to evaluate and improve the forecasting ability of models in the presence of instabilities, which are widespread in economic time series. Empirically relevant examples include predicting the financial crisis of 2007-08, as well as, more broadly, fluctuations in asset prices, exchange rates, output growth, and inflation. In the context of unstable environments, I discuss how to assess models' forecasting ability; how to robustify models' estimation; and how to correctly report measures of forecast uncertainty. Importantly, and perhaps surprisingly, breaks in models' parameters are neither necessary nor sufficient to generate time variation in models' forecasting performance: thus, one should not test for breaks in models' parameters, but rather evaluate their forecasting ability in a robust way. In addition, local measures of models' forecasting performance are more appropriate than traditional, average measures. (JEL C51, C53, E31, E32, E37, F37)

Automated summary

This article discusses methods for evaluating and improving forecasting models in the presence of instabilities in economic time series. It emphasizes that breaks in models' parameters are not necessary or sufficient for generating time variation in forecasting performance, and suggests focusing on robust evaluation methods rather than testing for parameter breaks. Additionally, it highlights the importance of using local measures of forecasting performance over traditional average measures.