Conventional models overestimate the statistical significance of volume-outcome associations, compared with multilevel models

Objective: To compare the use of conventional statistical models with multilevel regression models in volume-outcome analyses of surgical procedures in an empirical case study. Study Design and Setting: Using conventional regression models and multilevel regression models, we estimated the effect of hospital volume and surgeon volume on 30-day mortality and length of postoperative hospital stay in persons who had an esophagectomy, pancreaticoduodenectomy, or major lung resection for cancer in Ontario, Canada, from 1994 to 1999. Results: The point estimates of volume-outcome associations were similar using either method; however, the 95% confidence intervals estimated by multilevel models were wider than those estimated by conventional models. A significant association between volume and mortality was identified in 2 of 18 (11%) comparisons using conventional analysis but in none of the 18 (0%) comparisons using multilevel analysis. and between volume and length of stay in 15 of 18 (83%) comparisons using conventional analysis and in 1 of 18 (6%) comparisons using multilevel analysis. Conclusion: Conventional and multilevel statistical models can yield substantially different results in the analysis of volume-outcome associations for surgical procedures. (c) 2005 Elsevier Inc. All rights reserved.