Is the ACS-NSQIP Risk Calculator Accurate in Predicting Adverse Postoperative Outcomes in the Emergency Setting? An Italian Single-center Preliminary Study

Background The ACS-NSQIP surgical risk calculator (SRC) is an open-access online tool that estimates the chance for adverse postoperative outcomes. The risk is estimated based on 21 patient-related variables and customized for specific surgical procedures. The purpose of this monocentric retrospective study is to validate its predictive value in an Italian emergency setting. Methods From January to December 2018, 317 patients underwent surgical procedures for acute cholecystitis (n = 103), appendicitis (n = 83), gastrointestinal perforation (n = 45), and intestinal obstruction (n = 86). Patients' personal risk was obtained and divided by the average risk to calculate a personal risk ratio (RR). Areas under the ROC curves (AUC) and Brier score were measured to assess both the discrimination and calibration of the predictive model. Results The AUC was 0.772 (95%CI 0.722-0.817,p < 0.0001; Brier 0.161) for serious complications, 0.887 (95%CI 0.847-0.919,p < 0.0001; Brier 0.072) for death, and 0.887 (95%CI 0.847-0.919,p < 0.0001; Brier 0.106) for discharge to nursing or rehab facility. Pneumonia, cardiac complications, and surgical site infection presented an AUC of 0.794 (95%CI 0.746-0.838,p < 0.001; Brier 0.103), 0.836 (95%CI 0.790-0.875,p < 0.0001; Brier 0.081), and 0.729 (95%CI 0.676-0.777,p < 0.0001; Brier 0.131), respectively. A RR > 1.24, RR > 1.52, and RR > 2.63 predicted the onset of serious complications (sensitivity = 60.47%, specificity = 64.07%; NPV = 81%), death (sensitivity = 82.76%, specificity = 62.85%; NPV = 97%), and discharge to nursing or rehab facility (sensitivity = 80.00%, specificity = 69.12%; NPV = 95%), respectively. Conclusions The calculator appears to be accurate in predicting adverse postoperative outcomes in our emergency setting. A RR cutoff provides a much more practical method to forecast the onset of a specific type of complication in a single patient.