Development and validation of self-monitoring auto-updating prognostic models of survival for hospitalized COVID-19 patients

Despite rapid and significant changes during the pandemic, prognostic models for COVID-19 patients do not currently account for data drifts. Here, the authors develop a framework for continuously monitoring and updating prognostic models and applied it to predict 28-day survival in COVID-19 patients. Clinical prognostic models can assist patient care decisions. However, their performance can drift over time and location, necessitating model monitoring and updating. Despite rapid and significant changes during the pandemic, prognostic models for COVID-19 patients do not currently account for these drifts. We develop a framework for continuously monitoring and updating prognostic models and apply it to predict 28-day survival in COVID-19 patients. We use demographic, laboratory, and clinical data from electronic health records of 34912 hospitalized COVID-19 patients from March 2020 until May 2022 and compare three modeling methods. Model calibration performance drift is immediately detected with minor fluctuations in discrimination. The overall calibration on the prospective validation cohort is significantly improved when comparing the dynamically updated models against their static counterparts. Our findings suggest that, using this framework, models remain accurate and well-calibrated across various waves, variants, race and sex and yield positive net-benefits.

Automated summary

This study develops a framework for continuously monitoring and updating prognostic models for COVID-19 patients, and applies it to predict patient survival. The results show that dynamically updating the models significantly improves their overall calibration performance.