Verifying model performance using validation of Pareto solutions

Calibrating a hydrological model using more than one independent data set (multi-objective calibration) can potentially improve parameter estimation. However, it often identifies multiple parameter sets whose combined performance is generally indistinguishable, as visualized via a Pareto front. The objective of this paper is to identify how Pareto optimal solutions perform during model validation, and to identify how objective function tradeoffs in calibration might shift along the Pareto front during validation. Model calibration was based on an ecohydrological model simulating leaf area index (LAI), evapotranspiration (ET) and streamflow. Focusing on a predominantly forested Australian catchment and using satellite derived LAI and ET with observed streamflow for calibration, gave understanding of differences in model behavior related to observed data. The performance shifts between validation and calibration periods were assessed for combinations of ET and streamflow and LAI and streamflow. For both observed data combinations the performance deteriorated between calibration to validation and the best performing validation parameter solutions were scattered along the calibration Pareto front. In general, the streamflow/LAI data pairing showed a greater deterioration in streamflow performance in validation compared to the streamflow/ET data pairing. However, in both cases there was significant parameter equifinality. The results highlight that model structural limitations are a greater issue for model validation performance than in calibration. The identification of one optimal parameter set from a Pareto set remains elusive.

Automated summary

Calibrating a hydrological model using multiple independent data sets can improve parameter estimation, but often leads to indistinguishable performance among parameter sets. This study investigates the performance of Pareto optimal solutions during model validation and the tradeoffs between objective functions during calibration. The ecohydrological model used focuses on a forested Australian catchment and simulates leaf area index, evapotranspiration, and streamflow. The results show that the performance deteriorated between calibration and validation, with no clear optimal parameter set identified from the Pareto set.