期刊
STATISTICS IN MEDICINE
卷 40, 期 19, 页码 4230-4251出版社
WILEY
DOI: 10.1002/sim.9025
关键词
binary outcomes; calibration; discrimination; external validation; minimum sample size; multivariable prediction model; net benefit
类别
资金
- Cancer Research UK [C49297/A27294]
- European Union's Horizon 2020 Research and Innovation Programme [825746]
- National Institute for Health Research School for Primary Care Research (NIHR SPCR)
- Netherlands Organisation for Health Research and Development [91617050]
- NIHR Biomedical Research Centre, Royal Marsden NHS Foundation Trust/Institute of Cancer Research, Oxford
- NIHR Biomedical Research Centre, Oxford
External validation is crucial in examining the performance of prediction models, but current studies often face issues with small sample sizes. To address this, determining the minimum sample size needed for a new external validation study with precise estimation calculations is proposed, taking into account calibration, discrimination, and clinical utility measures.
In prediction model research, external validation is needed to examine an existing model's performance using data independent to that for model development. Current external validation studies often suffer from small sample sizes and consequently imprecise predictive performance estimates. To address this, we propose how to determine the minimum sample size needed for a new external validation study of a prediction model for a binary outcome. Our calculations aim to precisely estimate calibration (Observed/Expected and calibration slope), discrimination (C-statistic), and clinical utility (net benefit). For each measure, we propose closed-form and iterative solutions for calculating the minimum sample size required. These require specifying: (i) target SEs (confidence interval widths) for each estimate of interest, (ii) the anticipated outcome event proportion in the validation population, (iii) the prediction model's anticipated (mis)calibration and variance of linear predictor values in the validation population, and (iv) potential risk thresholds for clinical decision-making. The calculations can also be used to inform whether the sample size of an existing (already collected) dataset is adequate for external validation. We illustrate our proposal for external validation of a prediction model for mechanical heart valve failure with an expected outcome event proportion of 0.018. Calculations suggest at least 9835 participants (177 events) are required to precisely estimate the calibration and discrimination measures, with this number driven by the calibration slope criterion, which we anticipate will often be the case. Also, 6443 participants (116 events) are required to precisely estimate net benefit at a risk threshold of 8%. Software code is provided.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据