Methodology of Performance Scoring in the d2 Sustained-Attention Test: Cumulative-Reliability Functions and Practical Guidelines

We provide a psychometric analysis of commonly used performance indices of the d2 sustained-attention test, and give methodical guidelines and recommendations, based on this research. We examined experimental effects of repeated testing on performance speed and accuracy (omission and commission errors), and further evaluated aspects of test reliability by means of cumulative reliability function (CRF) analysis. These aspects were also examined for a number of alternative (yet commonly used) scoring techniques and valuation methods. Results indicate that performance is sensitive to change, both differentially within (time-on-task) and between (test-retest) sessions. These effects did not severely affect test reliability, since perfect score reliability was observed for measures of speed (and was even preserved with half the test length) while variability and error scores were more problematic with respect to reliability. Notably, limitations particularly hold for commission but less so for omission errors. Our recommendations to researchers and practitioners are that (a) only the speed score (and error-corrected speed score) is eligible for highly reliable assessment, that (b) error scores might be used as a secondary measure (e.g., to check for aberrant behavior), that (c) variability scores might not be used at all. Given the exceptional reliability of performance speed, and (d) test length may be reduced up to 50%, if necessary for time-economic reasons, to serve purposes of population screening and field assessment. Public Significance Statement This study found that some commonly used scoring techniques to assess sustained-attention performance are problematic with regard to measurement accuracy (test reliability). It further advances the use of cumulative reliability function analysis for purposes of comparably evaluating tests or scoring techniques. The recommendation to practitioners is that only scores of average performance speed (but none of variability) are eligible for highly reliable psychometric assessment of basic cognitive functioning.