Design, methods, and reporting of impact studies of cardiovascular clinical prediction rules are suboptimal: a systematic review

Objectives: To evaluate design, methods, and reporting of impact studies of cardiovascular clinical prediction rules (CPRs). Study Design and Setting: We conducted a systematic review. Impact studies of cardiovascular CPRs were identified by forward citation and electronic database searches. We categorized the design of impact studies as appropriate for randomized and nonrandomized experiments, excluding uncontrolled before-after study. For impact studies with appropriate study design, we assessed the quality of methods and reporting. We compared the quality of methods and reporting between impact and matched control studies. Results: We found 110 impact studies of cardiovascular CPRs. Of these, 65 (59.1%) used inappropriate designs. Of 45 impact studies with appropriate design, 31 (68.9%) had substantial risk of bias. Mean number of reporting domains that impact studies with appropriate study design adhered to was 10.2 of 21 domains (95% confidence interval, 9.3 and 11.1). The quality of methods and reporting was not clearly different between impact and matched control studies. Conclusion: We found most impact studies either used inappropriate study design, had substantial risk of bias, or poorly complied with reporting guidelines. This appears to be a common feature of complex interventions. Users of CPRs should critically evaluate evidence showing the effectiveness of CPRs. (c) 2021 Elsevier Inc. All rights reserved.

Automated summary

The study evaluated the design, methods, and reporting of impact studies of cardiovascular clinical prediction rules (CPRs). Most impact studies were found to either use inappropriate study design, have substantial risk of bias, or poorly adhere to reporting guidelines. The quality of methods and reporting did not show significant differences between impact and matched control studies. Users of CPRs should critically assess the evidence of the effectiveness of CPRs.