Seeing what we want to see: Confirmation bias in animal behavior research

Confirmation bias is the tendency of observers to see what they expect to see while conducting scientific research. Although confirmation bias has been well-studied by psychologists in the context of qualitative judgments, it has been much less studied with respect to the kinds of quantitative observations made by behavioral biologists. We carried out two experiments that used multiple observers of the aggression and foraging behaviors of red-backed salamanders (Plethodon cinereus) to determine whether behavioral observations were influenced by the a priori expectations of observers. In both experiments, one group of observers was given a specific set of expectations with respect to sex differences in salamander behavior, while a second group was given the opposite set of expectations. In one experiment, observers collected data on variable sets of live salamanders, while in the other experiment, observers collected data from identical videotaped trials. Across experiments and observed behaviors, the expectations of observers did appear to bias observations, but only to a small or moderate degree. Confirmation bias never accounted for more than 13% of the observed variation in behavior, and was generally equivalent to < 20% of the mean value of each variable. The estimated magnitude of confirmation bias was quite similar for men and women, suggesting no relationship between observer gender and susceptibility to confirmation bias. We believe that these results are largely optimistic with respect to confirmation bias in behavioral ecology, in that they suggest the bias may often be small relative to individual variation in behavior, even for relatively inexperienced observers.