Physics constrained learning for data-driven inverse modeling from sparse observations

Deep neural networks (DNN) can model nonlinear relations between physical quantities. Those DNNs are embedded in physical systems described by partial differential equations (PDE) and trained by minimizing a loss function that measures the discrepancy between predictions and observations in some chosen norm. This loss function often includes the PDE constraints as a penalty term when only sparse observations are available. As a result, the PDE is only satisfied approximately by the solution. However, the penalty term typically slows down the convergence of the optimizer for stiff problems. We present a new approach that trains the embedded DNNs while numerically satisfying the PDE constraints. We developed an algorithm that enables differentiating both explicit and implicit numerical solvers in reverse-mode automatic differentiation. Our method allows the gradients of the DNNs and the PDE solvers to be computed in a unified framework. We demonstrate that our approach enjoys faster convergence and better stability in relatively stiff problems compared to the penalty method. Our approach could solve and accelerate a wide range of data-driven inverse modeling, where the physical constraints are described by PDEs and need to be satisfied accurately. (C) 2022 Elsevier Inc. All rights reserved.

Automated summary

This article presents a new approach that trains deep neural networks (DNNs) while numerically satisfying partial differential equation (PDE) constraints. The algorithm developed allows differentiation of both explicit and implicit numerical solvers in reverse-mode automatic differentiation. The approach demonstrates faster convergence and better stability in relatively stiff problems compared to the penalty method.