Handling over-dispersion of performance indicators

Objectives: A problem can arise when a performance indicator shows substantially more variability than would be expected by chance alone, since ignoring such ''over-dispersion'' could lead to a large number of institutions being inappropriately classified as ''abnormal''. A number of options for handling this phenomenon are investigated, ranging from improved risk stratification to fitting a statistical model that robustly estimates the degree of over-dispersion. Design: Retrospective analysis of publicly available data on survival following coronary artery bypass grafts, emergency readmission rates, and teenage pregnancies. Setting: NHS trusts in England. Results: Funnel plots clearly show the influence of the method chosen for dealing with over-dispersion on the ''banding'' a trust receives. Both multiplicative and additive approaches are feasible and give intuitively reasonable results, but the additive random effects formulation appears to have a stronger conceptual foundation. Conclusion: A random effects model may offer a reasonable solution. This method has now been adopted by the UK Healthcare Commission in their derivation of star ratings.