Does risk adjustment of the CMS quality measures for nursing homes matter?

Background: The Centers for Medicare and Medicaid Services (CMS) publish a report card for nursing homes with 19 clinical quality measures (QMs). These measures include minimal risk adjustment. Objectives: To develop QMs with more extensive risk adjustment and to investigate the impact on quality rankings. Research Design: Retrospective analysis of individual level data reported in the Minimum Data Set (MDS). Random effect logistic models were used to estimate risk adjustment models for 5 outcomes: pressure ulcers for high and low risk patients, physical restraints, and pain for long- and short-stay patients. These models were used to create 5 QMs with extended risk adjustment, enhanced QMs (EQMs). The EQMs were compared with the corresponding QMs. Subjects: All (17,469) nursing homes that reported MDS data in the period 2001-2005, and their 9.6 million residents. Measures: QMs were compared with EQMs for all nursing homes in terms of agreement on outlier identification: Kappa, false positive and false negative error rates. Results: Kappa values ranged from 0.63 to 0.90. False positive and negative error rates ranged from 8% to 37%. Agreement between QMs and EQMs was better on high quality rather than on low quality. Conclusions: More extensive risk adjustment changes quality ranking of nursing homes and should be considered as potential improvement to the current QMs. Other methodological issues related to construction of the QMs should also be investigated to determine if they are important in the context of nursing home care.