Predictive Capability Assessment of Probabilistic Machine Learning Models for Density Prediction of Conventional and Synthetic Jet Fuels

Machine Learning (ML) models are increasingly applied in the field of jet fuel property predictions due to their ability of modeling a high number of complex composition-property relationships directly on measurement data. Their applicability is still limited as for safety relevant use cases like synthetic fuel approval or aircraft design consequences of prediction errors might be too severe to be acceptable. For Machine Learning algorithms, the predictive capability strongly depends on the data utilized for the training of the models. Predictions for fuels that differ from the training data might have uncertainties that need to be systematically considered. We present an approach of utilizing the probabilistic ML algorithm Gaussian Process Regression to model jet fuel properties and estimate the uncertainty that results from limited training data. We apply this approach for the example of density over a range of -40 to 140 degrees C. To assess the influence of synthetic fuels on the predictive capability, two models are studied, one trained exclusively on conventional fuels data and the other one trained on the same conventional fuels and additional synthetic fuels. To quantify the predictive capability of the models, we introduce three metrics that measure the accuracy and precision of the prediction as well as the validity and reliability of the estimated prediction interval. Results show that prediction intervals can correctly be estimated by both models and a valid estimation of the predictive capability is possible. Furthermore, the addition of synthetic fuels data drastically improves the accuracy, reduces the uncertainty, and is necessary to achieve adequate predictions for the considered hold-out fuels.

Automated summary

Machine Learning models are increasingly used in jet fuel property predictions, but their applicability is limited for safety relevant use cases. It is essential to systematically consider data and use algorithms such as Gaussian Process Regression to estimate the prediction uncertainty.