Overall assessment for selected markers from high-throughput data

Reproducibility, a hallmark of science, is typically assessed in validation studies. We focus on high-throughput studies where a large number of biomarkers is measured in a training study, but only a subset of the most significant findings is selected and re-tested in a validation study. Our aim is to get the statistical measures of overall assessment for the selected markers, by integrating the information in both the training and validation studies. Naive statistical measures, such as the combined P$$ P $$-value by conventional meta-analysis, that ignore the non-random selection are clearly biased, producing over-optimistic significance. We use the false-discovery rate (FDR) concept to develop a selection-adjusted FDR (sFDR) as an overall assessment measure. We describe the link between the overall assessment and other concepts such as replicability and meta-analysis. Some simulation studies and two real metabolomic datasets are considered to illustrate the application of sFDR in high-throughput data analyses.

Automated summary

This paper focuses on reproducibility assessment in high-throughput studies and proposes a selection-adjusted false-discovery rate (sFDR) as an overall assessment measure. By integrating information from both training and validation studies and considering the effects of non-random selection, sFDR provides a more accurate evaluation. Simulation studies and real metabolomic datasets are used to illustrate the application of sFDR in high-throughput data analysis.