4.6 Article

A multi-scale investigation of the human communication system's response to visual disruption

期刊

ROYAL SOCIETY OPEN SCIENCE
卷 9, 期 4, 页码 -

出版社

ROYAL SOC
DOI: 10.1098/rsos.211489

关键词

kinematics; gesture; speech; compensation; multimodal

向作者/读者索取更多资源

This study finds that in human communication, speech can compensate when the visual channel is disrupted, contradicting the notion that gesture plays a secondary role. Both gesture and speech dynamically adapt to meet communicative needs, highlighting the flexible nature of multi-modal communicative behaviors.
In human communication, when the speech is disrupted, the visual channel (e.g. manual gestures) can compensate to ensure successful communication. Whether speech also compensates when the visual channel is disrupted is an open question, and one that significantly bears on the status of the gestural modality. We test whether gesture and speech are dynamically co-adapted to meet communicative needs. To this end, we parametrically reduce visibility during casual conversational interaction and measure the effects on speakers' communicative behaviour using motion tracking and manual annotation for kinematic and acoustic analyses. We found that visual signalling effort was flexibly adapted in response to a decrease in visual quality (especially motion energy, gesture rate, size, velocity and hold-time). Interestingly, speech was also affected: speech intensity increased in response to reduced visual quality (particularly in speech-gesture utterances, but independently of kinematics). Our findings highlight that multi-modal communicative behaviours are flexibly adapted at multiple scales of measurement and question the notion that gesture plays an inferior role to speech.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.6
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据