A framework of incorporating confidence levels to deal with uncertainty in pairwise comparisons

Pairwise comparison is a key ingredient in multi-criteria decision analysis. The method is based on a set of comparisons conducted by a group of experts, comparing all possible pairs of alternatives involved in the decision process. The outcome is the estimation of weights determining the ranking of alternatives. In this paper, we introduce a new framework for the incorporation of confidence levels in pairwise comparisons, in order to deal with uncertainty issues related to the individual expert judgments. We discuss how the confidence levels can be related to the probability of rank reversal by introducing a theoretical model based on the multivariate normal cumulative distribution function. A comparison between theoretical and numerical results (Monte Carlo simulations), reveals a very good agreement. The proposed framework may provide a very good basis for pairwise comparison extensions aiming to provide further information regarding the accuracy for the evaluation of the final outcome.

Automated summary

This paper introduces a new framework for incorporating confidence levels in pairwise comparisons to address uncertainty issues related to individual expert judgments. The theoretical model based on the multivariate normal cumulative distribution function is compared with numerical results (Monte Carlo simulations) showing good agreement. The proposed framework may provide a basis for extensions aiming to provide further information on the accuracy of the evaluation of the final outcome.