New relevance and significance measures to replace p-values

The p-value has been debated exorbitantly in the last decades, experiencing fierce critique, but also finding some advocates. The fundamental issue with its misleading interpretation stems from its common use for testing the unrealistic null hypothesis of an effect that is precisely zero. A meaningful question asks instead whether the effect is relevant. It is then unavoidable that a threshold for relevance is chosen. Considerations that can lead to agreeable conventions for this choice are presented for several commonly used statistical situations. Based on the threshold, a simple quantitative measure of relevance emerges naturally. Statistical inference for the effect should be based on the confidence interval for the relevance measure. A classification of results that goes beyond a simple distinction like significant / non-significant is proposed. On the other hand, if desired, a single number called the secured relevance may summarize the result, like the p-value does it, but with a scientifically meaningful interpretation.

Automated summary

The concept of p-value has been widely debated in recent decades, with criticism over its interpretation of testing unrealistic null hypotheses. The focus should shift towards determining the relevance of an effect, leading to the proposal of a relevant threshold and quantitative measure. Statistical inference should be based on confidence intervals for relevance, which could potentially lead to a more comprehensive classification of results beyond simply significant or non-significant findings. If desired, a single number called the secured relevance may provide a scientifically meaningful interpretation of the results.