The methodological quality of economic evaluations of guideline implementation into clinical practice:: A systematic review of empiric studies

Objectives: Despite the emphasis on efficiency of health-care services delivery, there is an imperfect evidence base to inform decisions about whether and how to develop and implement guidelines into clinical practice. In general, studies evaluating the economics of guideline implementation lack methodological rigor. We conducted a systematic review of empiric studies to assess advances in the economic evaluations of guideline implementation. Methods: The Cochrane Effective Professional and Organisational Change Group specialized register and the MEDLINE database were searched for English publications between January 1998 and July 2004 that reported objective effect measures and implementation costs. We extracted data on study characteristics, quality of study design, and economic methodology. It was assessed whether the economic evaluations followed methodological guidance. Results: We included 24 economic evaluations, involving 21 controlled trials and three interrupted time series designs. The studies involved varying settings, targeted professionals, targeted behaviors, clinical guidelines, and implementation strategies. Overall, it was difficult to determine the quality of study designs owing to poor reporting. In addition, most economic evaluations were methodologically flawed: studies did not follow guidelines for evaluation design, data collection, and data analysis. Conclusions: The increasing importance of the value for money of providing health care seems to be reflected by an increase in empiric economic evaluations of guideline implementation. Because of the heterogeneity and poor methodological quality of these studies, however, the resulting evidence is still of limited use in decision-making. There seems to be a need for more methodological guidance, especially in terms of data collection and data synthesis, to appropriately evaluate the economics of developing and implementing guidelines into clinical practice.