Evidence of Lack of Treatment Efficacy Derived From Statistically Nonsignificant Results of Randomized Clinical Trials

IMPORTANCE Many randomized clinical trials yield statistically nonsignificant results. Such results are difficult to interpret within the dominant statistical framework. OBJECTIVE To estimate the strength of evidence in favor of the null hypothesis of no effect vs the prespecified effectiveness hypothesis among nonsignificant primary outcome results of randomized clinical trials by application of the likelihood ratio. DESIGN, SETTING, AND PARTICIPANTS Cross-sectional study of statistically nonsignificant results for primary outcomes of randomized clinical trials published in 6 leading general medical journals in 2021. OUTCOME MEASURES The likelihood ratio for the null hypothesis of no effect vs the effectiveness hypothesis stated in the trial protocol (alternate hypothesis). The likelihood ratio quantifies the support that the data provide to one hypothesis vs the other. RESULTS In 130 articles that reported 169 statistically nonsignificant results for primary outcomes, 15 results (8.9%) favored the alternate hypothesis (likelihood ratio, <1), and 154 (91.1%) favored the null hypothesis of no effect (likelihood ratio, >1). For 117 (69.2%), the likelihood ratio exceeded 10; for 88 (52.1%), it exceeded 100; and for 50 (29.6%), it exceeded 1000. Likelihood ratioswere onlyweakly correlated with P values (Spearman r, 0.16; P =.045). CONCLUSIONS A large proportion of statistically nonsignificant primary outcome results of randomized clinical trials provided strong support for the hypothesis of no effect vs the alternate hypothesis of clinical efficacy stated a priori. Reporting the likelihood ratio may improve the interpretation of clinical trials, particularly when observed differences in the primary outcome are statistically nonsignificant.

Automated summary

This study aims to estimate the strength of evidence for the null hypothesis vs the effectiveness hypothesis in statistically nonsignificant primary outcome results of randomized clinical trials. The results demonstrate that many trials provide strong support for the null hypothesis of no effect.