Reservoir computing dissection and visualization based on directed network embedding

The reservoir computing (RC) has recently gained considerable attention in practice and many methods have been developed to study its internal mechanism. However, the specific role played by the reservoir nodes of RC in time series prediction is still to be defined. An interpretable RC model wherein its reservoir network is designated as the directed acyclic network (DAN) is proposed with focus on time series prediction in this paper. In virtue of asymmetric transitivity and hierarchical structure of DAN, we present a directed network embedding method to identify the latent memory property of each node in the DAN. Such memory property is utilized to characterize the roles played by the reservoir nodes on the prediction performance of the RC. Meanwhile, it can also be leveraged to identify the corresponding memory community of DAN. As a result, we demonstrate how the reservoir network structure takes effect on the prediction performance from the perspective of memory community. In addition, two novel hyperparameters with the deterministic meaning are introduced to quantify the influence of the model initialization on the reservoir input so as to facilitate further dissection of the interpretable RC. The experimental results indicate that tuning these hyperparameters, which is explicable in terms of the Taylor expansion of the activation function, serves as an essential step in achieving superior prediction performance. Finally, comparative experiments with some other RC models on various time series benchmarks are also conducted. (c) 2021 Elsevier B.V. All rights reserved.

Automated summary

This paper proposes an interpretable reservoir computing model based on a directed acyclic network (DAN), which identifies memory properties of reservoir nodes in time series prediction and analyzes the impact of reservoir network structure on prediction performance from the perspective of memory community. Two novel hyperparameters with deterministic meaning are introduced to quantify the influence of model initialization on reservoir input, which significantly contributes to achieving superior prediction performance.