Measuring severe maternal morbidity: validation of potential measures

BACKGROUND: Both maternal mortality rate and severe maternal morbidity rate have risen significantly in the United Sates. Recently, the Centers for Disease Control and Prevention introduced International Classification of Diseases, 9th revision, criteria for defining severe maternal morbidity with the use of administrative data sources; however, those criteria have not been validated with the use of chart reviews. OBJECTIVE: The primary aim of the current study was to validate the Centers for Disease Control and Prevention International Classification of Diseases, 9th revision, criteria for the identification of severe maternal morbidity. This analysis initially required the development of a reproducible set of clinical conditions that were judged to be consistent with severe maternal morbidity to be used as the clinical gold standard for validation. Alternative criteria for severe maternal morbidity were also examined. STUDY DESIGN: The 67,468 deliveries that occurred during a 12-month period from 16 participating California hospitals were screened initially for severe maternal morbidity with the presence of any of 4 criteria: (1) Centers for Disease Control and Prevention International Classification of Diseases, 9th revision, diagnosis and procedure codes; (2) prolonged postpartum length of stay (>3 standard deviations beyond the mean length of stay for the California population); (3) any maternal intensive care unit admissions (with the use of hospital billing sources); and (4) the administration of any blood product (with the use of transfusion service data). Complete medical records for all screen-positive cases were examined to determine whether they satisfied the criteria for the clinical gold standard (determined by 4 rounds of a modified Delphi technique). Descriptive and statistical analyses that included area under the receiver operating characteristic curve and C-statistic were performed. RESULTS: The Centers for Disease Control and Prevention International Classification of Diseases, 9th revision, criteria had a reasonably high sensitivity of 0.77 and a positive predictive value of 0.44 with a C-statistic of 0.87. The most important source of false-positive cases were mothers whose only criterion was 1-2 units of blood products. The Centers for Disease Control and Prevention International Classification of Diseases, 9th revision, criteria screen rate ranged from 0.51-2.45% among hospitals. True positive severe maternal morbidity ranged from 0.05-1.13%. When hospitals were grouped by their neonatal intensive care unit level of care, severe maternal morbidity rates were statistically lower at facilities with lower level neonatal intensive care units (P < .0001). CONCLUSION: The Centers for Disease Control and Prevention International Classification of Diseases, 9th revision, criteria can serve as a reasonable administrative metric for measuring severe maternal morbidity at population levels. Caution should be used with the use of these criteria for individual hospitals, because case-mix effects appear to be strong.