Failed Attempts to Improve the Reliability of the Alcohol Visual Probe Task Following Empirical Recommendations

The visual probe task (VPT) is a computerized task used to measure attentional bias to substance-related stimuli. Little research has examined the psychometric properties of the VPT, despite concerns it demonstrates poor test-retest reliability and internal consistency. These issues can reduce confidence in inferences based on VPT performance. As such, we attempted to identify parameters under which the reliability of the alcohol VPT might be improved by applying recent empirical recommendations for outlier handling, bias calculation, and task design from the anxiety literature. We reanalyzed data from 3 previously published studies in our laboratory and 2 newly collected data sets. We compared tasks which presented images on the left/right of the screen to above/below, whether participants responded to the location or content of the probe, and whether general alcohol-related images or images personalized to the individual were used. In each VPT we also applied a priori outlier removal (2 and 3 standard deviations and median absolute difference) and data-driven outlier removal (winsorizing), in addition to calculating trial-level bias scores. Across all studies and tasks internal consistency and test-retest reliability of attentional bias measures were inadequate. There was no consistent improvement in internal consistency or test-retest reliability as a function of outlier removal methods. We were unable to demonstrate adequate reliability of the alcohol VPT, which further supports observations that these tasks may not yield reliable measures. Future research should focus on improving the reliability of these tasks or abandoning them in favor of more reliable alternatives.