Best (but often forgotten) Statistical Practices: Measuring Real-World Intervention Effectiveness using Electronic Health Data

The evidence base supporting the use of most interventions consists primarily of data from randomized controlled trials (RCTs), but how and to whom interventions are delivered in clinical practice may differ substantially from these foundational RCTs. With the increasing availability of electronic health data, it is now feasible to study the real-world effectiveness of a wide range of interventions. However, real-world intervention effectiveness studies using electronic health data face many challenges including data quality, selection bias, confounding by indication, and lack of generalizability. In this article, we describe the key barriers to generating high-quality evidence from real-world intervention effectiveness studies and suggest statistical best practices for addressing them.

Automated summary

The use of most interventions is primarily supported by evidence from randomized controlled trials (RCTs), but there may be substantial differences in the delivery of interventions in clinical practice compared to these foundational RCTs. With the increasing availability of electronic health data, it is now possible to study the real-world effectiveness of a wide range of interventions. However, real-world intervention effectiveness studies using electronic health data face challenges such as data quality, selection bias, confounding by indication, and lack of generalizability. In this article, we discuss the barriers to generating high-quality evidence from real-world intervention effectiveness studies and suggest statistical best practices to address these challenges.