Unpacking the black box: Examining the (de)Gender categorization effect in human-machine communication

Although studies have explored the gender categorization effect in both face-to-face and mediated communication environments in relation to the use of gender-linked language, whether the effect still holds in the context of human-machine communication (HMC) remains unknown. To examine this question, in this study, we asked 245 participants to assign gender categories to targets after viewing transcripts of the targets conversations with a chatbot and a human interlocutor. The results showed that the participants had a better-than-chance probability (68.98%) of correctly guessing the gender of the target based on the target-human conversation transcripts. However, the predictive power of the language cues decreased sharply to a less-than-chance level (42.86%) based on target-chatbot conversation transcripts. We also examined the roles that social media use and demographics played in the gender categorization processes in both computer-mediated communication and HMC contexts. Although far from conclusive, our results suggested that there were significant differences between the styles of conversation in the target-chatbot and target-human interlocutor transcripts. These findings imply that people use different approaches when communicating with human and non-human interlocutors.