Lip movements enhance speech representations and effective connectivity in auditory dorsal stream

A B S T R A C T Viewing speaker's lip movements facilitates speech perception, especially under adverse listening conditions, but the neural mechanisms of this perceptual benefit at the phonemic and feature levels remain unclear. This fMRI study addressed this question by quantifying regional multivariate representation and network organization underlying audiovisual speech-in-noise perception. Behaviorally, valid lip movements improved recognition of place of articulation to aid phoneme identification. Meanwhile, lip movements enhanced neural representations of phonemes in left auditory dorsal stream regions, including frontal speech motor areas and supramarginal gyrus (SMG). Moreover, neural representations of place of articulation and voicing features were promoted differentially by lip movements in these regions, with voicing enhanced in Broca's area while place of articulation better encoded in left ventral premotor cortex and SMG. Next, dynamic causal modeling (DCM) analysis showed that such local changes were accompanied by strengthened effective connectivity along the dorsal stream. Moreover, the neurite orientation dispersion of the left arcuate fasciculus, the bearing skeleton of auditory dorsal stream, predicted the visual enhancements of neural representations and effective connectivity. Our findings provide novel insight to speech science that lip movements promote both local phonemic and feature encoding and network connectivity in the dorsal pathway and the functional enhancement is mediated by the microstructural architecture of the circuit.

Automated summary

This fMRI study investigated the neural mechanisms of the perceptual benefit of lip movements during speech perception. The results showed that lip movements improved recognition of phonemic features and enhanced neural representations in the left auditory dorsal stream regions. Different regions were found to encode voicing and place of articulation features differently in response to lip movements. Dynamic causal modeling analysis revealed strengthened effective connectivity along the dorsal stream. Furthermore, the microstructural architecture of the left arcuate fasciculus was found to predict the visual enhancements of neural representations and effective connectivity. These findings provide novel insights into the role of lip movements in speech perception.