Profound effect of study design factors on ventilator-associated pneumonia incidence of prevention studies: benchmarking the literature experience

Background: The ventilator-associated pneumonia incident proportion (VAP-IP) is highly variable among control groups of studies of methods for its prevention. The objective here is to develop and validate a literature-derived benchmark against which these groups can be profiled. Methods: A literature search yielded 95 cohort groups and control and intervention groups of 150 studies of either non-antimicrobial or antimicrobial methods of VAP prevention. The 95 cohort groups comprise a benchmark set (30 groups), from which the reference funnel plot (RFP) was derived, and a search set (65 groups), against which the benchmark was validated. The VAP-IP data of the benchmark set were found in five published systematic reviews, whereas the VAP-IP data of the search set were abstracted directly from the literature. Findings: Among the 95 cohort groups, the VAP-IP of groups with size > 399 was significantly lower than the VAP-IP of smaller groups. Compared with the RFP, 15 of 51 (29%) control groups from studies of antimicrobial methods of VAP prevention with concurrent design were high outlier versus 2 of 110 (2%) control groups from other types of study design (P < 0.001). There were only 22 (14%) outlier groups, all low outlier, among the 162 intervention groups. Conclusions: Study design factors such as concurrency and study size have potentially greater influence on the VAP-IP than do the VAP prevention methods under study. The outlier status of control groups were inapparent in the individual studies and the meta-analyses and yet would have confounded the estimates of treatment effect.