A scattered landscape: assessment of the evidence base for 71 patient decision aids developed in a hospital setting

Background Recent publications reveal shortcomings in evidence review and summarization methods for patient decision aids. In the large-scale Share to Care (S2C) Shared Decision Making (SDM) project at the University Hospital Kiel, Germany, one of 4 SDM interventions was to develop up to 80 decision aids for patients. Best available evidence on the treatments' impact on patient-relevant outcomes was systematically appraised to feed this information into the decision aids. Aims of this paper were to (1) describe how PtDAs are developed and how S2C evidence reviews for each PtDA are conducted, (2) appraise the quality of the best available evidence identified and (3) identify challenges associated with identified evidence. Methods The quality of the identified evidence was assessed based on GRADE quality criteria and categorized into high-, moderate-, low-, very low-quality evidence. Evidence appraisal was conducted across all outcomes assessed in an evidence review and for specific groups of outcomes, namely mortality, morbidity, quality of life, and treatment harms. Challenges in evidence interpretation and summarization resulting from the characteristics of decision aids and the type and quality of evidence are identified and discussed. Results Evidence reviews assessed on average 25 systematic reviews/guidelines/studies and took about 3 months to be completed. Despite rigorous review processes, nearly 70% of outcome-specific information derived for decision aids was based on low-quality and mostly on non-directly comparative evidence. Evidence on quality of life and harms was often not provided or not in sufficient form/detail. Challenges in evidence interpretation for use in decision aids resulted from, e.g., a lack of directly comparative evidence or the existence of very heterogeneous evidence for the diverse treatments being compared. Conclusions Evidence reviews in this project were carefully conducted and summarized. However, the evidence identified for our decision aids was indeed a scattered landscape and often poor quality. Facing a high prevalence of low-quality, non-directly comparative evidence for treatment alternatives doesn't mean it is not necessary to choose an evidence-based approach to inform patients. While there is an urgent need for high quality comparative trials, best available evidence nevertheless has to be appraised and transparently communicated to patients.

Automated summary

This study aims to describe the development process of patient decision aids and evaluate the quality of identified evidence. The study found that despite careful evidence evaluation and summarization, the quality of identified evidence was generally poor and faced various challenges.