How far can the uncertainty on a Digital Soil Map be known?: A numerical experiment using pseudo values of clay content obtained from Vis-SWIR hyperspectral imagery

Digital Soil Map uncertainty is usually evaluated from a set of independent soil observations that are used to determine various uncertainty indicators. However, the number and locations of the sites that constitute these evaluations may impact the value of these indicators. In this paper, a numerical experiment on uncertainty indicators was performed using the pseudo values of topsoil clay content obtained from an airborne hyperspectral image in the Cap Bon region (Tunisia). These pseudo values form a soil pattern with a large extent (46% of 300 km(2)), high resolution (5 m) and good accuracy (R-val(2) = 0.75) while being free of visible artefacts and pedologically plausible. Therefore, the dataset was considered a fair representation of reality while providing a quasi-unlimited choice of sites. The numerical experiment considered three Quantile Regression Forests as examples of DSM models by using inputs from relief soil covariates and geographical locations that were calibrated from 200, 2000 and 100,000 individuals respectively (low, medium and high quality models). Their uncertainty indicators were first evaluated by calculating four uncertainty indicators (ME, MSE, SSMSE and PICP) from a large independent validation set of 100,000 sites. These uncertainty indicators were then computed from independent evaluation sets of different sizes (from 50 to 500 sites) and from different locations (500 evaluation sets of each size). The independent evaluation sets were selected following a stratified random sampling using compact geographical strata. The numerical experiment showed that the values of the uncertainty indicators were highly variable across numbers and locations of sites. The largest variations were observed for evaluation sets with fewer than 100 sites, but non-negligible variations remained for larger evaluation datasets. This result suggested that evaluations from independent sets convey a non-negligible error on the uncertainty indicators, which increases as the number of sites decrease. Evaluations of DSM models from independent evaluation sets should be interpreted with care and uncertainty on validation results should be systematically estimated. For that, numerical experiments for benchmarking DSM models on known soil patterns across the world would be a valuable complement to the analytical expressions for the uncertainty indicators and the many DSM applications for which these analytical expressions are not valid. This would serve also to improve the sampling techniques for the calibration and evaluation datasets to reduce the error when estimating the uncertainty of a DSM product.