期刊
ISCIENCE
卷 25, 期 12, 页码 -出版社
CELL PRESS
DOI: 10.1016/j.isci.2022.105711
关键词
-
资金
- Studienstiftung des Deutschen Volkes
- Deutsche Forschungsgemeinschaft (DFG) [Schw 511/25-1, SPP 2392]
This study investigates the effects of time-synchronized facial information on vocal emotion recognition (VER) and finds that cochlear implant (CI) users perform worse in emotion classification tasks compared to individuals with normal hearing. The study also finds that CI users show larger benefits to VER when facial information is congruent with the auditory information, suggesting that they compensate for their auditory impairment through crossmodal integration.
Speech comprehension counts as a benchmark outcome of cochlear implants (CIs)-disregarding the communicative importance of efficient integration of audiovisual (AV) socio-emotional information. We investigated effects of time-synchronized facial information on vocal emotion recognition (VER). In Experiment 1, 26 CI users and normal-hearing (NH) individuals classified emotions for auditory-only, AV congruent, or AV incongruent utterances. In Experiment 2, we compared crossmodal effects between groups with adaptive testing, calibrating auditory difficulty via voice morphs from emotional caricatures to anti-caricatures. CI users performed lower than NH individuals, and VER was correlated with life quality. Importantly, they showed larger benefits to VER with congruent facial emotional information even at equal auditory-only performance levels, suggesting that their larger crossmodal benefits result from deafness-related compensation rather than degraded acoustic representations. Crucially, vocal caricatures enhanced CI users' VER. Findings advocate AV stimuli during CI rehabilitation and suggest perspectives of caricaturing for both perceptual trainings and sound processor technology.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据