Cross-validation aggregation for combining autoregressive neural network forecasts

This paper evaluates k-fold and Monte Carlo cross-validation and aggregation (crogging) for combining neural network autoregressive forecasts. We introduce Monte Carlo crogging which combines bootstrapping and cross-validation (CV) in a single approach through repeated random splitting of the original time series into mutually exclusive datasets for training. As the training/validation split is independent of the number of folds, the algorithm offers more flexibility in the size, and number of training samples compared to k-fold cross-validation. The study also provides for crogging and bagging: (1) the first systematic evaluation across time series length and combination size, (2) a bias and variance decomposition of the forecast errors to understand improvement gains, and (3) a comparison to established benchmarks of model averaging and selection. Crogging can easily be extended to other autoregressive models. Results on real and simulated series demonstrate significant improvements in forecasting accuracy especially for short time series and long forecast horizons. (C) 2016 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved.