Comparing the reliability of responses to telephone-administered versus self-administered web-based surveys in a case-control study of adult malignant brain cancer

Introduction: To determine whether a Web-based survey was an acceptable method of data collection for a clinic-based case-control study of adult brain cancer, the authors compared the reliability of paired responses to a main and resurvey for participants completing surveys by telephone (n = 74) or self-administered on the Web (n = 465) between 2003 and 2006. Methods: Recruitment of cases was done at the Evanston Northwestern Healthcare Kellogg Cancer Care Center and the Duke University Medical Center Cancer Control division, and controls were friends and siblings of cases. Twenty-five variables were examined, including smoking, oral contraceptive and residential histories, water sources, meat preparation, fruit and vegetable consumption, and pesticide use. Weighted and simple kappa's were estimated for categorical and binary variables, respectively. Results: The number of concordant paired responses was summed for use in linear regression. Respondents were 97% White and 85% had postsecondary education. Kappa's for individual questions ranged from 0.31 (duration of residence in a single family house) to 0.96 (ever smoked), with a median of 0.57 (95% confidence interval, 0.47-0.64). The median number of concordant responses was 16.2 (range, 5-22). Reliability was greater for controls than cases, Web-based versus telephone responders, females, and higher-income responders. Frequency of e-mail and Internet use was not associated with reliability. Conclusions: A self-administered, Web-based survey was a feasible and appropriate mode of interview in this study. The comparable reliability of Web compared with telephone responses suggest that Web-based self-interviews could be a cost-effective alternative to traditional modes of interview.