Reliability of body condition scoring of sheep for cross-farm assessments

In order to use body condition scoring as a cross-farm assessment tool, it is important that different assessors can consistently grade sheep along the same scale. This paper examined the intra- and inter-assessor reliability of three trained and experienced assessors who independently assessed the body condition of 141 Lleyn, Cambridge-cross and Welsh Mule-crossbred ewes using full- or half-point scoring precision of a six-point ordinal scale. Assessor reliability was evaluated using percentage (%) agreement, weighted kappa (kappa(w)), and Kendall's coefficient of concordance (W). Kruskal-Wallis one-way analysis of variance and graphical representation of the data were also used to examine for assessor bias. The on-farm studies found that the intra-assessor reliability of an assessor, who provided training in the scoring method, was higher when half- (96%, kappa(w) 0.7, W 0.7) rather than full-unit scoring precision (79%, kappa(w) 0.6, W 0.6) was used. Similarly, a higher level of inter-observer agreement was found when two additional assessors applied half- (94%, kappa(w) 0.6, W 0.7) rather than full-unit scores (93%, kappa(w) 0.4, W 0.4). Consequently, the effect of a brief re-calibration exercise on the between-observer agreement for the assessment of full-unit body condition scores (BCS) was examined. Prior to the exercise, the paired agreement between two assessors and the trainer ranged from 68 to 78%, kappa(w) 0.3-0.5, and W 0.4-0.5. Following the re-calibration exercise, the level of inter-observer agreement increased to 75-93%, kappa(w) 0.4-0.7, and W 0.4-0.6. No significant effect of assessor bias was found (p > 0.05). However, most sample sheep were identified within the mid-range of body condition (BCS 2-3), which affected the analysis and interpretation of reliability data. Overall, the results suggested that trained and experienced assessors reliably scored the body condition of sheep using both half- and full-unit scores, and that a period of re-calibration may offer a feasible means of maintaining the consistency of cross-farm assessments performed by different assessors. (C) 2011 Published by Elsevier B.V.