Out-of-system Care and Recording of Patient Characteristics Critical for Comparative Effectiveness Research

Background: It is unclear how out-of-system care or electronic health record (EHR) discontinuity (i.e., receiving care outside of an EHR system) may affect validity of comparative effectiveness research using these data. We aimed to compare the misclassification of key variables in patients with high versus low EHR continuity. Methods: The study cohort comprised patients ages >= 65 identified in electronic health records from two US provider networks linked with Medicare insurance claims data from 2007 to 2014. By comparing electronic health records and claims data, we quantified EHR continuity by the proportion of encounters captured by the EHRs (i.e., capture proportion). Within levels of EHR continuity, for 40 key variables, we quantified misclassification by mean standardized differences between coding based on EHRs alone versus linked claims and EHR data. Results: Based on 183,739 patients, we found that mean capture proportion in a single electronic health record system was 16%-27% across two provider networks. Patients with highest level of EHR continuity (capture proportion >= 80%) had 11.4- to 17.4-fold less variable misclassification, when compared with those with lowest level of EHR continuity (capture proportion<10%). Capturing at least 60% of the encounters in an EHR system was required to have reasonable variable classification (mean standardized difference <0.1). We found modest differences in comorbidity profiles between patients with high and low EHR continuity. Conclusions: EHR discontinuity may lead to substantial misclassification in key variables. Restricting comparative effectiveness research to patients with high EHR continuity may confer a favorable benefit (reducing information bias) to risk (losing generalizability) ratio.