How to write about alternatives to classical hypothesis testing outside of the statistical literature: Approximate Bayesian model selection applied to a biomechanics study

By now, statisticians and the broader research community are aware of the controversies surrounding traditional hypothesis testing and p values. Many alternative viewpoints and methods have been proposed, as exemplified by The American Statistician's recent special issue themed World beyond p<0.05. However, it seems clear that the broader scientific effort may benefit if alternatives to classical hypothesis testing are described in venues beyond the statistical literature. This paper addresses two relevant gaps in statistical practice. First, we describe three principles statisticians and their collaborators can use to publish about alternatives to classical hypothesis testing in the literature outside of statistics. Second, we describe an existing BIC-based approximation to Bayesian model selection as a complete alternative approach to classical hypothesis testing. This approach is easy to conduct and interpret, even for analysts who do not have fully Bayesian expertise in analyzing data. Perhaps surprisingly, it does not appear that the BIC approximation has yet been described in the context of World beyond p<0.05. We address both gaps by describing a recent collaborative effort where we used the BIC-based techniques to publish a paper about hypothesis testing alternatives in a high-end biomechanics journal.

Automated summary

This paper discusses the controversies surrounding classical hypothesis testing and p values, and proposes principles for describing alternative methods in venues beyond the statistical literature. It also presents an existing BIC-based approximation to Bayesian model selection as a complete alternative approach to classical hypothesis testing.