Is the Social Desirability Effect in Human-Robot Interaction overestimated? A Conceptual Replication Study Indicates Less Robust Effects

The Computers are social actors (CASA) assumption (Nass and Moon in J Soc Issues 56:81-103, 2000. https://doi.org/10.1111/0022-4537.00153) states that humans apply social norms and expectations to technical devices. One such norm is to distort one's own response in a socially desirable direction during interviews. However, findings for such an effect are mixed in the literature. Therefore, a new study on the effect of social desirability bias in human-robot evaluation was conducted, aiming for a conceptual replication of previous findings. In a between-subject laboratory experiment, N=107 participants had to evaluate the robot and the interaction quality after a short conversation in three different groups: In one group, the evaluation was conducted using (1) the same robot of the former interaction, (2) a different robot, (3) a tablet computer. According to the CASA assumption, it was expected, that evaluations on likability and quality of interaction, are higher in the condition with the same robot conducting the evaluation, compared to a different robot or a tablet computer because robots are treated as social actors and hence humans distort ratings in a socially desirable direction. Based on previous findings, we expected robots to evoke higher anthropomorphism and feelings of social presence compared to the tablet computer as potential explanation. However, the data did not support the hypotheses. Low sample size, low statistical power, lack of measurement validation and other problems that could lead to an overestimation of effect sizes-in this study and the literature in general-are discussed in light of the replicability crisis.

Automated summary

The study aimed to investigate the impact of social desirability bias in human-robot evaluation, expecting higher ratings for likability and interaction quality when evaluating the same robot based on the CASA assumption. However, the data did not support the hypotheses, highlighting potential issues with sample size, statistical power, and measurement validation.