Stabilized neural ordinary differential equations for long-time forecasting of dynamical systems

In data-driven modeling of spatiotemporal phenomena careful consideration is needed in capturing the dynamics of the high wavenumbers. This problem becomes especially challenging when the system of interest exhibits shocks or chaotic dynamics. We present a data-driven modeling method that accurately captures shocks and chaotic dynamics by proposing a new architecture, stabilized neural ordinary differential equation (ODE). In our proposed architecture, we learn the right-hand-side (RHS) of an ODE by adding the outputs of two NN together where one learns a linear term and the other a nonlinear term. Specifically, we implement this by training a sparse linear convolutional NN to learn the linear term and a dense fully-connected nonlinear NN to learn the nonlinear term. This contrasts with the standard neural ODE which involves training a single NN for the RHS. We apply this setup to the viscous Burgers equation, which exhibits shocked behavior, and show stabilized neural ODEs provide better short-time tracking, prediction of the energy spectrum, and robustness to noisy initial conditions than standard neural ODEs. We also apply this method to chaotic trajectories of the Kuramoto-Sivashinsky equation. In this case, stabilized neural ODEs keep long-time trajectories on the attractor, and are highly robust to noisy initial conditions, while standard neural ODEs fail at achieving either of these results. We conclude by demonstrating how stabilizing neural ODEs provide a natural extension for use in reduced-order modeling by projecting the dynamics onto the eigenvectors of the learned linear term.

Automated summary

Careful consideration is needed in capturing the dynamics of high wavenumbers in data-driven modeling of spatiotemporal phenomena, especially when shocks or chaotic dynamics are present. To address this challenge, a new architecture called stabilized neural ordinary differential equation (ODE) is proposed, which accurately captures shocks and chaotic dynamics. By combining the outputs of two neural networks, one learning the linear term and the other the nonlinear term, the proposed architecture learns the right-hand-side (RHS) of the ODE. Experimental results on the viscous Burgers equation and the Kuramoto-Sivashinsky equation demonstrate that stabilized neural ODEs outperform standard neural ODEs in short-time tracking, prediction of energy spectrum, robustness to noisy initial conditions, and long-time trajectory keeping on the attractor.