A comprehensive uncertainty analysis of the analytic hierarchy process methodology applied in the context of environmental decision making

An overview of uncertainty issues associated with the analytic hierarchy process (AHP) is provided. Further, an explicit understanding of uncertainty (designation, categorization, and quantification) with respect to the methodological properties of the AHP is developed and used to analyse a hypothetical group decision problem located in the context of environmental decision making (EDM). To calculate the numerical impact of especially designed uncertainty scenarios (USs) on the final ranking given by the AHP, a simulation experiment is conducted using R. It evaluates the impact of uncertainty within three variants of a hypothetical decision-making case by calculating an overall uncertainty measure. The consideration of uncertainty may lead to a rank reversal in comparison with that analysis neglecting uncertainty (best alternative given by the AHP). The results show that the absolute maximal impact caused by a US is approximately 0.03. With respect to a single US and the specific case characteristics, in about 50% of the simulated runs a rank reversal occurs. Similar shares of rank reversal over different USs within a single variant of the case raise the question to which uncertainty should be given prior attention in decision-making practice. For decision analysts in EDM, this result implies that additional resources may be necessary to commonly negotiate with decision makers that uncertainties should be addressed. From a theoretical normative point of view, the effects of considering uncertainty issues in the AHP methodology cannot satisfy the ideal of a rational decision analysis. From a descriptive point of view, considering the practice of decision makers, the impacts of the considered uncertainties stay within reasonable limits, meaning that the maximal numerical impact stays on the hundredths decimal place.