Modified echo state network for prediction of nonlinear chaotic time series

In this paper, we focus on the prediction issue of the nonlinear chaotic time series. In particular, we introduce the modified echo state network (M-ESN) to predict the time series of nonlinear chaotic system. Thereinto, to solve the ill-conditioned output weight matrix caused by vast neurons in hidden layer of the ESN, we introduce the hybrid regularized network (HRN) based on the l(2/3) regularization and the l(2) regularization. Thereinto, the l(2/3) regularization plays the role to produce the sparse output weight and import the oracle property to the prediction model, while the l(2) regularization plays the role to shrink the amplitude of the output weight matrix produced by the former to further improve the generalization ability of the HRN. Therefore, the M-ESN produces the sparse output weight matrix, enjoys the oracle property and has well generalization ability for the nonlinear chaotic time series. Besides, considering that the random input weight matrix may affect the whole system performance, we have investigated the plasticity property of the neuron and developed a fine-tuning strategy of the input weight matrix based on the Bienenstock, Cooper, Munro (BCM) theory. The input weight matrix in M-ESN is fine-tuned in the unsupervised training process. We introduce and explain its calculation process in detail. Finally, we evaluate the M-ESN by the time series of the classical nonlinear Lorenz chaotic system, the time series of the nonlinear Lv chaotic system, the time series of the Hindmarsh-Rose neuron model and the time series of the rainfall series in Hefei city in past 50 years. The simulation results indicate the M-ESN has well one-step and multistep prediction performances for nonlinear chaotic time series.

Automated summary

This paper introduces the modified echo state network (M-ESN) and the hybrid regularized network (HRN) for predicting nonlinear chaotic time series, showing that M-ESN can generate sparse output weight matrices with good generalization ability.