Magnitude of Potential Biases in a Simulated Case-Control Study of the Effectiveness of Influenza Vaccination

Background. Many influenza vaccine effectiveness estimates have been made using case-control methods. Although several forms of bias may distort estimates of vaccine effectiveness derived from case-control studies, there have been few attempts to quantify the magnitude of these biases. Methods. We estimated the magnitude of potential biases in influenza vaccine effectiveness values derived from case-control studies from several factors, including bias from differential use of diagnostic testing based on influenza vaccine status, imperfect diagnostic test characteristics, and confounding. A decision tree model was used to simulate an influenza vaccine effectiveness case-control study in children. Using probability distributions, we varied the value of factors that influence vaccine effectiveness estimates, including diagnostic test characteristics, vaccine coverage, likelihood of receiving a diagnostic test for influenza, likelihood that a child hospitalized with acute respiratory infection had influenza, and others. Bias was measured as the difference between the effectiveness observed in the simulated case-control study and a true underlying effectiveness value. Results and Conclusions. We found an average difference between observed and true vaccine effectiveness of -11.9%. Observed vaccine effectiveness underestimated the true effectiveness in 88% of model iterations. Diagnostic test specificity exhibited the strongest association with observed vaccine effectiveness, followed by the likelihood of receiving a diagnostic test based on vaccination status and the likelihood that a child hospitalized with acute respiratory infection had influenza. Our findings suggest that the potential biases in case-control studies that we examined tend to result in underestimates of true influenza vaccine effects.