Physics-based Penalization for Hyperparameter Estimation in Gaussian Process Regression

In Gaussian Process Regression (GPR), hyperparameters are often estimated by maximizing the marginal likelihood function. However, this data-dominant hyperparameter estimation process can lead to poor extrapolation performance and often violates known physics, especially in sparse data scenarios. In this paper, we embed physics-based knowledge through penalization of the marginal likelihood objective function and study the effect of this new objective on consistency of optimal hyperparameters and quality of GPR fit. Three case studies are presented, where physics-based knowledge is available in the form of linear Partial Differential Equations (PDEs), while initial or boundary conditions are not known so direct forward simulation of the model is challenging. The results reveal that the new hyperparameter set obtained from the augmented marginal likelihood function can improve the prediction performance of GPR, reduce the violation of the underlying physics, and mitigate overfitting problems.

Automated summary

In Gaussian Process Regression (GPR), embedding physics-based knowledge through penalization of the marginal likelihood function can improve the prediction performance, reduce violation of known physics, and mitigate overfitting problems. This paper presents three case studies where physics-based knowledge is available in the form of linear Partial Differential Equations (PDEs), and shows that the new hyperparameter set obtained from the augmented marginal likelihood function leads to consistent optimal hyperparameters and quality GPR fit, despite the challenge of unknown initial or boundary conditions.