Can a Bayesian belief network for survival prediction in patients with extremity metastases (PATHFx) be exter-nally validated in an Asian cohort of 356 surgically treated patients?

Background and purpose - Predicted survival may influence the treatment decision for patients with skeletal extremity metastasis, and PATHFx was designed to predict the likelihood of a patient dying in the next 24 months. How -ever, the performance of prediction models could have eth- nogeographical variations. We asked if PATHFx generalized well to our Taiwanese cohort consisting of 356 surgically treated patients with extremity metastasis.Patients and methods - We included 356 patients who underwent surgery for skeletal extremity metastasis in a tertiary center in Taiwan between 2014 and 2019 to validate PATHFx's survival predictions at 6 different time points. Model performance was assessed by concordance index (c-index), calibration analysis, decision curve analysis (DCA), Brier score, and model consistency (MC).Results - The c-indexes for the 1-, 3-, 6-, 12-, 18-, and 24-month survival estimations were 0.71, 0.66, 0.65, 0.69, 0.68, and 0.67, respectively. The calibration analysis demon- strated positive calibration intercepts for survival predictions at all 6 timepoints, indicating PATHFx tended to underesti- mate the actual survival. The Brier scores for the 6 models were all less than their respective null model's. DCA demon- strated that only the 6-, 12-, 18-, and 24-month predictions appeared useful for clinical decision-making across a wide range of threshold probabilities. The MC was < 0.9 when the 6-and 12-month models were compared with the 12-month and 18-month models, respectively.Interpretation - In this Asian cohort, PATHFx's per- formance was not as encouraging as those of prior valida- tion studies. Clinicians should be cognizant of the potential decline in validity of any tools designed using data outside their particular patient population. Developers of survival prediction tools such as PATHFx might refine their algo- rithms using data from diverse, contemporary patients that is more reflective of the world's population.

Automated summary

This study validates the performance of the PATHFx model in predicting survival for Taiwanese patients with skeletal extremity metastasis. The results showed that the performance of PATHFx was not as encouraging as previous validation studies. Clinicians should be aware of the potential decline in validity when using tools designed using data from different populations, and developers of survival prediction tools should consider using more diverse samples to improve their algorithms.