Universal Prediction Distribution for Surrogate Models

The use of surrogate models instead of computationally expensive simulation codes is very convenient in engineering. Roughly speaking, there are two kinds of surrogate models: the deterministic and the probabilistic ones. These last are generally based on Gaussian assumptions. The main advantage of the probabilistic approach is that it provides a measure of uncertainty associated with the surrogate model in the whole space. This uncertainty is an efficient tool to construct strategies for various problems such as prediction enhancement, optimization, or inversion. In this paper, we propose a universal method to define a measure of uncertainty suitable for any surrogate model either deterministic or probabilistic. It relies on cross-validation submodel predictions. This empirical distribution may be computed in much more general frames than the Gaussian one; thus it is called the universal prediction distribution (UP distribution). It allows the definition of many sampling criteria. We give and study adaptive sampling techniques for global refinement and an extension of the so-called efficient global optimization algorithm. We also discuss the use of the UP distribution for inversion problems. The performances of these new algorithms are studied both on toy models and on an engineering design problem.