A Narrative Review and Proposed Framework for Using Health System Data with Systematic Reviews to Support Decision-making

Systematic reviews are a necessary, but often insufficient, source of information to address the decision-making needs of health systems. In this paper, we address when and how the use of health system data might make systematic reviews more useful to decision-makers. We describe the different ways in which health system data can be used with systematic reviews, identify scenarios in which the addition of health system data may be most helpful (i.e., to improve the strength of evidence, to improve the applicability of evidence, and to inform the implementation of evidence), and discuss the importance of framing the limitations and considerations when using unpublished health system data in reviews. We developed a framework to guide the use of health system data alongside systematic reviews based on a narrative review of the literature and empirical experience. We also offer recommendations to improve the transparency of reporting when using health system data alongside systematic reviews including providing rationale for employing additional data, details on the data source, critical appraisal to understand study design biases as well as limitations in data and information quality, and how the unpublished data compares to the systematically reviewed data. Future methodological work on how best to handle internal and external validity concerns of health system data in the context of systematically reviewed data and work on developing infrastructure to do this type of work is needed.