The effect of administration mode on CAHPS survey response rates and results: A comparison of mail and web-based approaches

Objective The objective of this study was to compare response rates, respondents' characteristics, and substantive results for CAHPS surveys administered using web and mail protocols. Data Sources Patients who had one or more primary care visits in the preceding 6 months. Study Design/Data Collection Methods Patients for whom primary care practices had email addresses were randomized to one of four survey administration protocols: web via a portal invitation; web via an email invitation; combination of web and mail; and mail only. Another sample of patients without known email addresses was surveyed by mail. Samples of nonrespondents to the Internet and mail protocols were surveyed by telephone. Principal Findings Response rates to surveys administered using the Internet protocols were lower than for the surveys administered by mail (20 percent vs over 40 percent). However, characteristics of respondents and survey answers were very similar across protocols. Respondents without email addresses were older, less educated, and more likely to be male than those with email addresses, and there were a few differences in their responses. There was little evidence of nonresponse bias in either the mail or web protocols. Conclusion In this well-educated patient population, web protocols had lower response rates, but substantive results very similar to those from mail protocols.