TESTING ONE HYPOTHESIS MULTIPLE TIMES

In applied settings, hypothesis testing when a nuisance parameter is identifiable only under the alternative often reduces to a problem of testing one hypothesis multiple times (TOHM). Specifically, a fine discretization of the space of the nonidentifiable parameter is specified, and the null hypothesis is tested against a set of sub-alternative hypotheses, one for each point of the discretization. The resulting sub-test statistics are then combined to obtain a global p-value. We propose a computationally efficient inferential tool to perform TOHM under stringent significance requirements, such as those typically required in the physical sciences, (e.g., a p-value < 10(-7)). The resulting procedure leads to a generalized approach to performing inferences under nonstandard conditions, including non-nested model comparisons.

Automated summary

The study proposes a computationally efficient inferential tool for TOHM, allowing inferences to be made under stringent significance requirements, such as those in the physical sciences, and addressing nonstandard conditions like non-nested model comparisons. It offers a generalized approach to performing inferences under such conditions.