Processing of audiovisual stimuli in aphasic and non-brain-damaged listeners

Background: During speech perception not only auditory but also visual information (seen speech) is processed. This was shown by, for example, McGurk and MacDonald (1976). In their study participants watched videos showing a speaker articulating the syllable /ka/, while the sound of /pa/ was dubbed onto the video. The most commonly reported perception was neither of the two syllables, but a fusion of both (/ta/). This effect of audiovisual integration has been called the McGurk effect. Klitsch (2008) showed that the McGurk effect was also found for participants with aphasia and its strength was comparable to an age-matched control group. However her task was an offline measure and therefore provided limited information about the processing itself. Aims: In this study reaction time data will be analysed in addition to answer types. By this we aim to find out whether there are qualitative differences between healthy and aphasic processing, shown by different patterns in the reaction times. Methods & Procedures: Three aphasic participants and a group of fourteen non-brain-damaged control participants took part in a nonword identification task in which they were presented with a speaker pronouncing a syllable. Afterwards they had to choose the matching syllable from three written options. This task was carried out in four conditions: auditory only, visual only (articulatory movements), audiovisual, and McGurk (incongruent auditory and audiovisual information). Along with the answer types, the reaction times were also recorded for all participants. Outcomes & Results: The aphasic participants made more errors and were slower than the non-brain-damaged control participants across conditions. Within the McGurk condition we found different patterns for the non-brain-damaged and the aphasic participants: reaction times were increased for the control group whenever a McGurk-type answer was chosen (compared with answers representing the auditory or visual part of the stimulus), while there were no influences of the chosen answer type on the reaction times for the aphasic participants. Conclusions: Processing of audiovisual information was not only slower in three aphasic participants but also differed qualitatively from the non-brain-damaged control group. For the control participants reaction times differed depending on what answer type was chosen in the McGurk condition. This was not the case for either of the aphasic participants. We argue that the slow-down in undisturbed processing is caused by conscious access to the unimodal information, which might be lacking for the aphasic participants.