A tighter generalization bound for reservoir computing

While reservoir computing (RC) has demonstrated astonishing performance in many practical scenarios, the understanding of its capability for generalization on previously unseen data is limited. To address this issue, we propose a novel generalization bound for RC based on the empirical Rademacher complexity under the probably approximately correct learning framework. Note that the generalization bound for the RC is derived in terms of the model hyperparameters. For this reason, it can explore the dependencies of the generalization bound for RC on its hyperparameters. Compared with the existing generalization bound, our generalization bound for RC is tighter, which is verified by numerical experiments. Furthermore, we study the generalization bound for the RC corresponding to different reservoir graphs, including directed acyclic graph (DAG) and Erdos-Renyi undirected random graph (ER graph). Specifically, the generalization bound for the RC whose reservoir graph is designated as a DAG can be refined by leveraging the structural property (i.e., the longest path length) of the DAG. Finally, both theoretical and experimental findings confirm that the generalization bound for the RC of a DAG is lower and less sensitive to the model hyperparameters than that for the RC of an ER graph. Published under an exclusive license by AIP Publishing.

Automated summary

While reservoir computing has shown remarkable performance in various practical scenarios, its ability to generalize on unseen data is still limited. This paper proposes a novel generalization bound for reservoir computing, based on empirical Rademacher complexity, which explores the relationship between the model's performance and hyperparameters. The proposed bound is tighter and validated through numerical experiments. Additionally, the generalization bound for reservoir computing with a directed acyclic graph (DAG) is found to be lower and less sensitive to hyperparameters compared to that with an Erdos-Renyi undirected random graph (ER graph).