Equivalences of weighted kappas for multiple raters

Cohen's unweighted kappa and weighted kappa are popular descriptive statistics for measuring agreement between two raters on a categorical scale. With m >= 3 raters, there are several views in the literature on how to define agreement. We consider a family of weighted kappas for multiple raters using the concept of g-agreement (g = 2, 3, ... , m) which refers to the situation in which it is decided that there is agreement if g out of m raters assign an object to the same category. Given m raters, we may formulate m - 1 weighted kappas in this family, one for each type of g-agreement. We show that the m - 1 weighted kappas coincide if we use the weighting scheme proposed by Mielke et al. (2007) [31]. (C) 2011 Elsevier B.V. All rights reserved.