Journal
JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY
Volume 84, Issue 4, Pages 1105-1128Publisher
WILEY
DOI: 10.1111/rssb.12507
Keywords
FDR; FWER; infinite linear programming; multiple comparisons; optimal testing; strong control
Categories
Funding
- Israeli Science Foundation [2180/20]
Ask authors/readers for more resources
Multiple testing problems are important in modern statistical analysis, aiming to reject as many false null hypotheses as possible while controlling an overall measure of false discovery. This paper extends the optimal test for a single hypothesis to multiple testing problems and provides maximin rules for complex alternatives. The usefulness of these methods is demonstrated in numerical experiments and clinical trials, showing significant improvements in power.
Multiple testing problems (MTPs) are a staple of modern statistical analysis. The fundamental objective of MTPs is to reject as many false null hypotheses as possible (that is, maximize some notion of power), subject to controlling an overall measure of false discovery, like family-wise error rate (FWER) or false discovery rate (FDR). In this paper we provide generalizations to MTPs of the optimal Neyman-Pearson test for a single hypothesis. We show that for simple hypotheses, for both FWER and FDR and relevant notions of power, finding the optimal multiple testing procedure can be formulated as infinite dimensional binary programs and can in principle be solved for any number of hypotheses. We also characterize maximin rules for complex alternatives, and demonstrate that such rules can be found in practice, leading to improved practical procedures compared to existing alternatives that guarantee strong error control on the entire parameter space. We demonstrate the usefulness of these novel rules for identifying which studies contain signal in numerical experiments as well as in application to clinical trials with multiple studies. In various settings, the increase in power from using optimal and maximin procedures can range from 15% to more than 100%.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available