Hybrid Regularized Echo State Network for Multivariate Chaotic Time Series Prediction

Multivariate chaotic time series prediction is a hot research topic, the goal of which is to predict the future of the time series based on past observations. Echo state networks (ESNs) have recently been widely used in time series prediction, but there may be an ill-posed problem for a large number of unknown output weights. To solve this problem, we propose a hybrid regularized ESN, which employs a sparse regression with the L(1/2 )regularization and the L-2 regularization to compute the output weights. The L-1/2 penalty shows many attractive properties, such as unbiasedness and sparsity. The L-2 penalty presents appealing ability on shrinking the amplitude of the output weights. After the output weights are calculated, the input weights, internal weights, and output weights are fine-tuning by a Hessian-free optimization method-conjugate gradient back-propagation algorithm. The fine-tuning helps to bubble up the input information toward the output layer. Besides, the largest Lyapunov exponent is used to calculate the predictable horizon of a chaotic time series. Experimental results on benchmark and real-world datasets show that our proposed method is superior to other ESN-based models, as sparser, smaller-absolute-value, and more informative output weights are obtained. All of the predictions within the predictable horizon of the proposed model are accurate.