Diagnostic characteristics of clinical prediction models for obstructive sleep apnea in different clinic populations

As predictive factors and their diagnostic values are affected by the characteristics of the population studied, clinical prediction model for obstructive sleep apnea (OSA) may exhibit different diagnostic characteristics in different populations. We aimed to compare the diagnostic characteristics of clinical prediction models developed in two different populations. One hundred seventeen consecutive clinic patients (group 1) were evaluated to develop a clinical prediction model for OSA (local model). The diagnostic characteristics of this local model were compared with those of a foreign model by applying both models to another group of 52 patients who were referred to the same clinic (group 2). All patients underwent overnight polysomnography. The local model had an area under receiver operator characteristics curve of 79%. A cutoff of 0.6 was associated with sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of 77.9%, 72.5%, 84.5%, and 63.0%, respectively. The overall diagnostic accuracy, sensitivity, specificity, PPV, and NPV of the local model using data from patients in group 2 were 69.0%, 78.1%, 45.0%, 69.4%, and 56.3%, respectively. The foreign model had an overall diagnostic accuracy of 64.0% when applied to data from patients in group 2. At the optimal cutoff of 17, the foreign model was associated with sensitivity of 38.2%, specificity of 83.3%, NPV of 41.7% and PPV of 81.3%. Clinical prediction model for OSA derived from a foreign population exhibits markedly different diagnostic characteristics from one that is developed locally, even though the overall accuracy is similar. Our findings challenge the predictive usefulness and the external validity of clinical prediction models.