Revisiting the Basis of Sensitivity Analysis for Dynamical Earth System Models

This paper investigates the problem of global sensitivity analysis (GSA) of Dynamical Earth System Models and proposes a basis for how such analyses should be performed. We argue that (a) performance metric-based approaches to parameter GSA are actually identifiability analyses, (b) the use of a performance metric to assess sensitivity unavoidably distorts the information provided by the model about relative parameter importance, and (c) it is a serious conceptual flaw to interpret the results of such an analysis as being consistent and accurate indications of the sensitivity of the model response to parameter perturbations. Further, because such approaches depend on availability of system state/output observational data, the analysis they provide is necessarily incomplete. Here we frame the GSA problem from first principles, using trajectories of the partial derivatives of model outputs with respect to controlling factors as the theoretical basis for sensitivity, and construct a global sensitivity matrix from which statistical indices of total period time-aggregate parameter importance, and time series of time-varying parameter importance, can be inferred. We demonstrate this framework using the HBV-SASK conceptual hydrologic model applied to the Oldman basin in Canada and show that it disagrees with performance metric-based methods regarding which parameters exert the strongest controls on model behavior. Further, it is highly efficient, requiring less than 1,000 base samples to obtain stable and robust parameter importance assessments for our 10-parameter example. Plain Language Summary When developing and using computer-based models to (a) understand Earth and environmental systems, (b) make predictions, and/or (c) make management or policy decisions, it is very important to know which factors most strongly control the behaviors of the model. Tools to determine this are called sensitivity analysis (SA) methods. This paper shows that the use of model performance metrics to assess sensitivity is based in faulty reasoning. By framing the problem from first principles, a logical approach is developed that provides accurate and cost-effective assessments of both time-aggregate and time-varying parameter importance. Because the approach does not require availability of system output data, it enables a comprehensive assessment and can be applied to historical and predictive conditions, as well as to future scenarios.