Current Project
Audiovisual Processing in Aphasic and Non-Brain-Damaged Listeners: The Whole is More than the Sum of its Parts
Period: September 2007-September 2011
Lip-reading information has a big influence on human comprehension, as can be experienced every day. Thus lip-reading information undoubtedly helps healthy listeners to understand better under adverse (noisy) conditions, such as at a party or when several people are talking at the same time. However McGurk and MacDonald (1976) convincingly showed that lip-reading information is automatically integrated into speech comprehension even if conditions are perfect. Participants who heard the syllable /ba/ but simultaneously saw somebody articulating /ga/ perceived the syllable /da/ (McGurk effect). Furthermore lip-reading information was proven to have a beneficial influence on aphasic comprehension (cf. Buchman et al., 1986; Shindo et al., 1991). Studies concerning the McGurk effect in aphasia have shown that aphasic patients with comprehension disorders do exhibit a McGurk effect (Campbell et al., 1990; Youse et al., 2004).
The primary question investigated in this research project concerns the difference between the performance of aphasic participants in detecting minimal phonological differences and their brain responses. Thus it will be investigated whether aphasic patients’ brains might perceive differences that were not experienced consciously. Furthermore the influence of lip-reading information on these differences will be investigated, answering the question whether lip-reading information is integrated by the brain even if it does not influence the participant’s performance.
Before looking at the brain responses two behavioral studies will be conducted. Within the first of these, aphasic performance in discrimination tasks under the influence of additional (matching or non-matching) lip-reading information is assessed. It will be investigated in which ways the participants are distracted by non-matching or benefit from matching information, thus if the influence is bigger for one specific phonetic feature or if it concerns all three (manner of articulation, voicing or place of articulation) equally. The second study will be formed by a partial replication of previous results (Klitsch, 2008) about the McGurk effect in aphasia. This will give information about the way participants are subject to a McGurk percept. This information will be necessary to interpret the results of the third study on an individual basis.
Finally the main part of this research project is formed by an ERP-experiment that is based on mismatch negativity (MMN). MMN is a component that occurs when the brain perceives an auditory deviation from a standard sound. During an active oddball task participants (aphasic as well as healthy controls) will be asked to press a button whenever they hear a stimulus that does not match the standard. Simultaneously ERP data will be recorded and later analyzed for occurrence of MMN. Thus the occurrence of the McGurk effect in aphasia will be investigated with this objective technology, giving information about the integration of lip-reading information in the brains of aphasic subjects. Furthermore this technology will be used to investigate whether there is a difference between phonological differences aphasic people report to perceive (i.e. when they pushed the button) and those their brains do perceive (as seen by MMN).
Participants
Dörte Hessler
Dr. Roel Jonkers
Prof. Dr. Roelien Bastiaanse
Funding
This project is funded by an Ubbo Emmius Fellowship
References
Buchman, A., Garron, D. C., Trost-Cardamone, J. E., Wichter, M. D. and Schwartz, M. (1986). Word deafness: one hundred years later. Journal of Neurology, Neurosurgery and Psychiatry, 49 (5), 489-499.
Campbell, R., Garwood, J., Franklin, S., Howard, D., Landis, T. and Regard, M. (1990). Neuropsychological studies of auditory-visual fusion illusions. Four case studies and their implications. Neuropsychologia, 28 (8), 787-802.
Klitsch, J. (2008). Open your eyes and listen carefully. Auditory and audiovisual speech perception and the McGurk effect in Dutch speakers with and without aphasia. PhD-Thesis, University of Groningen.
McGurk, H. and MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264 (5588), 746-748.
Shindo, M., Kaga, K. and Tanaka, Y. (1991). Speech discrimination and lipreading in patients with word deafness or auditory agnosia. Brain and Language 40 (2), 153-161.
Youse, K. M., Cienkowski, K. M. and Coelho, C. A. (2004). Auditory-visual speech perception in an adult with aphasia. Brain Injury, 18 (8), 825-834.