Measuring provider adherence to tobacco treatment guidelines: A comparison of electronic medical record review, patient survey, and provider survey

An accurate method of measuring primary care providers' tobacco counseling actions is needed for monitoring adherence to clinical practice guidelines. We compared three methods of measuring providers' tobacco counseling practices: electronic medical record (EMR) review, patient survey, and provider survey. We mailed a survey to 1,613 smokers seen by 114 Boston-area primary care providers during a 2-month period to assess what tobacco counseling actions had occurred at the visit (N = 766; 47% response rate). Smokers' reports were compared with the EMR and with their providers' self-reported usual tobacco counseling practices, derived from a provider survey (N = 110; 96% response rate). Patients reported receiving each counseling action more frequently than providers documented it in the EMR. Agreement between the patient survey and the EMR was poor for all 5A steps (kappa statistic = 0.01-0.22). Providers reported that they often or always performed each 5A action at a higher rate than indicated by EMR or patient report. However, providers who said they often or always performed individual 5A steps did not have consistently higher mean rates of EMR documentation or patient report than those who said they performed the 5A's less frequently. Little agreement was found among the three methods of measuring primary care providers' tobacco counseling actions. Implementing an EMR does not necessarily improve providers' documentation of tobacco interventions, but EMR adaptations that would standardize provider documentation of tobacco counseling might make the EMR a more reliable tool for monitoring providers' delivery of tobacco treatment services.