Assessing the performance of spectroscopic models for cancer diagnostics using cross-validation and permutation testing

Multivariate classifiers (such as Linear Discriminant Analysis, Support Vector Machines etc) are known to be useful tools for making diagnostic decisions based on spectroscopic data. However, robust techniques for assessing their performance (e.g. by sensitivity and specificity) are vital if the application of these methods is to be successful in the clinic. In this work the application of repeated cross-validation for estimating confidence intervals for sensitivity and specificity of multivariate classifiers is presented. Furthermore, permutation testing is presented as a suitable technique for estimating the probability of obtaining the observed sensitivity and specificity by chance. Both approaches are demonstrated through their application to a Raman spectroscopic model of gastrointestinal cancer.