4.7 Article

Crossmodal benefits to vocal emotion perception in cochlear implant users

Journal

ISCIENCE
Volume 25, Issue 12, Pages -

Publisher

CELL PRESS
DOI: 10.1016/j.isci.2022.105711

Keywords

-

Funding

  1. Studienstiftung des Deutschen Volkes
  2. Deutsche Forschungsgemeinschaft (DFG) [Schw 511/25-1, SPP 2392]

Ask authors/readers for more resources

This study investigates the effects of time-synchronized facial information on vocal emotion recognition (VER) and finds that cochlear implant (CI) users perform worse in emotion classification tasks compared to individuals with normal hearing. The study also finds that CI users show larger benefits to VER when facial information is congruent with the auditory information, suggesting that they compensate for their auditory impairment through crossmodal integration.
Speech comprehension counts as a benchmark outcome of cochlear implants (CIs)-disregarding the communicative importance of efficient integration of audiovisual (AV) socio-emotional information. We investigated effects of time-synchronized facial information on vocal emotion recognition (VER). In Experiment 1, 26 CI users and normal-hearing (NH) individuals classified emotions for auditory-only, AV congruent, or AV incongruent utterances. In Experiment 2, we compared crossmodal effects between groups with adaptive testing, calibrating auditory difficulty via voice morphs from emotional caricatures to anti-caricatures. CI users performed lower than NH individuals, and VER was correlated with life quality. Importantly, they showed larger benefits to VER with congruent facial emotional information even at equal auditory-only performance levels, suggesting that their larger crossmodal benefits result from deafness-related compensation rather than degraded acoustic representations. Crucially, vocal caricatures enhanced CI users' VER. Findings advocate AV stimuli during CI rehabilitation and suggest perspectives of caricaturing for both perceptual trainings and sound processor technology.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available