A General Propensity Score for Signal Identification Using Tree-Based Scan Statistics

The tree-based scan statistic (TreeScan; Martin Kulldorff, Harvard Medical School, Boston, Massachusetts) is a data-mining method that adjusts for multiple testing of correlated hypotheses when screening thousands of potential adverse events for signal identification. Simulation has demonstrated the promise of TreeScan with a propensity score (PS)-matched cohort design. However, it is unclear which variables to include in a PS for applied signal identification studies to simultaneously adjust for confounding across potential outcomes. We selected 4 pairs of medications with well-understood safety profiles. For each pair, we evaluated 5 candidate PSs with different combinations of 1) predefined general covariates (comorbidity, frailty, utilization), 2) empirically selected (data-driven) covariates, and 3) covariates tailored to the drug pair. For each pair, statistical alerting patterns were similar with alternative PSs (<= 11 alerts in 7,996 outcomes scanned). Inclusion of covariates tailored to exposure did not appreciably affect screening results. Inclusion of empirically selected covariates can provide better proxy coverage for confounders but can also decrease statistical power. Unlike tailored covariates, empirical and predefined general covariates can be applied out of the box for signal identification. The choice of PS depends on the level of concern about residual confounding versus loss of power. Potential signals should be followed by pharmacoepidemiologic assessment where confounding control is tailored to the specific outcome(s) under investigation.

Automated summary

The study investigated the selection of propensity score (PS) and related covariates in signal identification research. The results suggest that the choice of PS depends on balancing residual confounding concerns with loss of power, and how to customize confounding control for specific outcomes.