Reliability and validity of the American Hospital Association's national longitudinal survey of health information technology adoption

Objective To evaluate the internal consistency, construct validity, and criterion validity of a battery of items measuring information technology (IT) adoption, included in the American Hospital Association (AHA) IT Supplement Survey. Methods We analyzed the 2012 release of the AHA IT Supplement Survey. We performed reliability analysis using Cronbach's alpha and part-whole correlations, construct validity analysis using principal component analysis (PCA), and criterion validity analysis by assessing the items' sensitivity and specificity of predicting attestation to Medicare Meaningful Use (MU). Results Twenty-eight items of the 31-item instrument and five of six functionality subcategories defined by the AHA all produced reliable scales (alpha's between 0.833 and 0.958). PCA mostly confirmed the AHA's categorization of functionalities; however, some items loaded only weakly onto the factor most associated with their survey category, and one category loaded onto two separate factors. The battery of items was a valid predictor of attestation to MU, producing a sensitivity of 0.82 and a specificity of 0.72. Discussion The battery of items performed well on most indices of reliability and validity. However, they lack some components of ideal survey design, leaving open the possibility that respondents are not responding independently to each item in the survey. Despite measuring only a portion of the objectives required for attestation to MU, the items are a moderately sensitive and specific predictor of attestation. Conclusions The analyzed instrument exhibits satisfactory reliability and validity.