Evaluating fitness-for-use of electronic health records in pragmatic clinical trials: reported practices and recommendations

Objective To empirically explore how pragmatic clinical trials (PCTs) that used real-world data (RWD) assessed study-specific fitness-for-use. Methods We conducted interviews and surveys with PCT teams who used electronic health record (EHR) data to ascertain endpoints. The survey cataloged key concerns about RWD, activities used to assess data fitness-for-use, and related barriers encountered by study teams. Patterns and commonalities across trials were used to develop recommendations for study-specific fitness-for-use assessments. Results Of 15 invited trial teams, 7 interviews were conducted. Of 31 invited trials, 15 responded to the survey. Most respondents had prior experience using RWD (93%). Major concerns about EHR data were data reliability, missingness or incompleteness of EHR elements, variation in data quality across study sites, and presence of implausible or incorrect values. Although many PCTs conducted fitness-for-use activities (eg, data quality assessments, 11/14, 79%), less than a quarter did so before choosing a data source. Fitness-for-use activities, findings, and resulting study design changes were not often publically documented. Overall costs and personnel costs were barriers to fitness-for-use assessments. Discussion These results support three recommendations for PCTs that use EHR data for endpoint ascertainment. Trials should detail the rationale and plan for study-specific fitness-for-use activities, conduct study-specific fitness-for-use assessments early in the prestudy phase to inform study design changes before the trial begins, and share results of fitness-for-use assessments and description of relevant challenges and facilitators. Conclusion These recommendations can help researchers and end-users of real-world evidence improve characterization of RWD reliability and relevance in the PCT-specific context.

Automated summary

The study aimed to explore how pragmatic clinical trials using real-world data assessed study-specific fitness-for-use. Concerns about EHR data included reliability, missingness, variation in data quality, and incorrect values. Most PCTs conducted fitness-for-use activities, but less than a quarter did so before choosing a data source. Overall costs and personnel costs were barriers to fitness-for-use assessments.