The methodological quality of individual participant data meta-analysis on intervention effects: systematic review

OBJECTIVE To assess the methodological quality of individual participant data (IPD) meta-analysis and to identify areas for improvement. DESIGN Systematic review. DATA SOURCES Medline, Embase, and Cochrane Database of Systematic Reviews. ELIGIBILITY CRITERIA FOR SELECTING STUDIES Systematic reviews with IPD meta-analyses of randomised controlled trials on intervention effects published in English. RESULTS 323 IPD meta-analyses covering 21 clinical areas and published between 1991 and 2019 were included: 270 (84%) were non-Cochrane reviews and 269 (84%) were published in journals with a high impact factor (top quarter). The IPD meta-analyses showed low compliance in using a satisfactory technique to assess the risk of bias of the included randomised controlled trials (43%, 95% confidence interval 38% to 48%), accounting for risk of bias when interpreting results (40%, 34% to 45%), providing a list of excluded studies with justifications (32%, 27% to 37%), establishing an a priori protocol (31%, 26% to 36%), prespecifying methods for assessing both the overall effects (44%, 39% to 50%) and the participant-intervention interactions (31%, 26% to 36%), assessing and considering the potential of publication bias (31%, 26% to 36%), and conducting a comprehensive literature search (19%, 15% to 23%). Up to 126 (39%) IPD meta-analyses failed to obtain IPD from 90% or more of eligible participants or trials, among which only 60 (48%) provided reasons and 21 (17%) undertook certain strategies to account for the unavailable IPD. CONCLUSIONS The methodological quality of IPD meta-analyses is unsatisfactory. Future IPD meta-analyses need to establish an a priori protocol with prespecified data syntheses plan, comprehensively search the literature, critically appraise included randomised controlled trials with appropriate technique, account for risk of bias during data analyses and interpretation, and account for unavailable IPD.

Automated summary

This study assessed the methodological quality of individual participant data (IPD) meta-analysis and found unsatisfactory compliance in assessing risk of bias, interpreting results, providing a list of excluded studies, establishing an a priori protocol, and other key aspects. Future IPD meta-analyses need to improve protocol establishment, literature search, bias assessment, and strategies for dealing with unavailable data.