Increasing the replicability for linear models via adaptive significance levels

We put forward an adaptive a (type I error) that decreases as the information grows for hypothesis tests comparing nested linear models. A less elaborate adaptation was presented in Perez and Pericchi (Stat Probab Lett 85:20-24, 2014) for general i.i.d. models. The calibration proposed in this paper may be interpreted as a Bayes-nonBayes compromise, of a simple translation of a Bayes factor on frequentist terms that leads to statistical consistency, and most importantly, it is a step toward statistics that promotes replicable scientific findings.

Automated summary

We propose an adaptive method (type I error) that reduces as information increases for hypothesis tests comparing nested linear models. A simpler adaptation was presented in Perez and Pericchi (Stat Probab Lett 85:20-24, 2014) for general i.i.d. models. The calibration proposed in this paper can be interpreted as a compromise between Bayesian and non-Bayesian approaches, achieving statistical consistency through a simple translation of the Bayes factor into frequentist terms and, more importantly, promoting replicable scientific findings.