Assessing the representational accuracy of data-driven models: The case of the effect of urban green infrastructure on temperature

Data-driven modelling with machine learning (ML) is already being used for predictions in environmental science. However, it is less clear to what extent data-driven models that successfully predict a phenomenon are representationally accurate and thus increase our understanding of the phenomenon. Besides empirical accuracy, we propose three criteria to indirectly assess the relationships learned by the ML algorithms and how they relate to a phenomenon under investigation: first, consistency of the outcomes with background knowledge; second, the adequacy of the measurements, datasets and methods used to construct a data-driven model; third, the robustness of interpretable machine learning analyses across different ML algorithms. We apply the three criteria with a case study modelling of the effect of different urban green infrastructure types on temperature and show that our approach improves the assessment of representational accuracy and reduces representational uncertainty, which can improve the understanding of modelled phenomena.

Automated summary

This study discusses the accuracy of using data-driven models for environmental science predictions and the methods for studying the phenomenon. It proposes three criteria to assess the relationships learned by machine learning algorithms to improve the understanding of modeled phenomena.