Consensus-Based Sub-Indicator Weighting Approach: Constructing Composite Indicators Compatible with Expert Opinion

The weighting of sub-indicators is a relevant problem in the composite indicators literature and impacts several fields of science. None of the existing weighting approaches, Equal-Weights, Data-Driven, and Participatory, is exempt from criticism. Specifically, weights obtained by the Participatory approach are associated with two frequent problems: assessments errors and international comparisons. Mainly, the assessments errors occur when the number of sub-indicators to be assessed is high, as it requires more cognitive effort from decision-makers. The problem of international comparison occurs because the weights of the sub-indicators reflect the specific characteristics of the countries and are not necessarily the same. Selecting experts who know the countries involved increases the impact of expert assessments on the results as the number of experts qualified to carry out the assessments decreases. These are common problems in composite indicators such as the Global Innovation Index, Multidimensional Poverty Index, Sustainable Development Goals Index, and Ease of Doing Business Index. This research presents solutions to these two problems. First, experts ordered seventeen sub-indicators by importance, decreasing the cognitive effort of the experts and the assessment errors that occur when the sub-indicators are assessed directly or compared in pairs. Second, the order of importance is converted into weights through six assessment format transformation functions. The deviant assessments are identified by the Concordance Correlation Coefficient and Intraclass Correlation Coefficient and excluded. Sub-indicators are weighted with a twenty-nine percent higher consensus degree, allowing the construction of composite indicators compatible with collective opinion.

Automated summary

The weighting of sub-indicators is a relevant problem in the composite indicators literature and impacts several fields of science. None of the existing weighting approaches, Equal-Weights, Data-Driven, and Participatory, is exempt from criticism. This research presents solutions to the problems of assessment errors and international comparisons in composite indicators by ordering sub-indicators by importance and converting the order into weights through assessment format transformation functions, resulting in higher consensus and compatibility with collective opinion.