Comparison between SAPS II and SAPS 3 in predicting hospital mortality in a cohort of 103 Italian ICUs. Is new always better?

More recent severity scores should be more reliable than older ones because they account for the improvement in medical care over time. To provide more insight into this issue, we compared the predictive ability of the Simplified Acute Physiology Score (SAPS) II and SAPS 3 (originally developed from data collected in 1991-1992 and 2002, respectively) on a sample of critically ill patients. This was a prospective observational study on 3,661 patients from 103 Italian intensive care units. Standardized mortality ratios (SMRs) were calculated. Assessment of calibration across risk classes was performed using the GiViTI calibration belt. Discrimination was evaluated by means of the area under the receiver operating characteristic analysis. Both scores were shown to discriminate fairly. SAPS 3 largely overpredicted mortality, more than SAPS II (SMR 0.63, 95 % CI 0.60-0.66 vs. 0.87, 95 % CI 0.83-0.91). This result was consistent and statistically significant across all risk classes for SAPS 3. SAPS II did not show relevant deviations from ideal calibration in the first two deciles of risk, whereas in higher-risk classes it overpredicted mortality. Both scores provided unreliable predictions, but unexpectedly the newer SAPS 3 turned out to overpredict mortality more than the older SAPS II.