An examination of the equivalence of web-based versus paper-and-pencil upward feedback ratings: Rater- and ratee-level analyses

In this study, 5,257 employees provided upward feedback ratings for 759 target managers who had the option of having their subordinates rate them using a traditional paper-and-pencil (opscan) response mode or using the company's intranet. Preliminary analyses showed mean online ratings were more favorable than were mean paper-and-pencil ratings (d =.38). The authors then conducted rater- and ratee-level hierarchical regression analyses in which they controlled for rater and ratee characteristics. Results showed that (a) the association between response mode and upward feedback ratings was largely artifactual, with response mode accounting for 1 % or less of the variance in upward feedback ratings after controlling for rater and ratee characteristics; (b) ratee characteristics accounted for much more of the variance in upward feedback ratings than did rater characteristics; and (c) response mode was unrelated to the number of missing responses.