Jeffery A. Jones
Brain activation during an audiovisual speech perception task: An fMRI study of the McGurk effect.
NeuroReport, 14, 1129-1133.
Jones, J. A. & Callan, D. E.
published: 2003 | Research publication | Jones Lab
fMRI was used to assess the relationship between brain activation
and the degree of audiovisual integration of speech information
during a phoneme categorization task. Twelve subjects heard a
speaker say the syllable /aba/ paired either with video of the speaker
saying the same consonant or a diĦerent one (/ava/). In order
to manipulate the degree of audiovisual integration, the audio
was either synchronous or 7400ms out of phase with the visual
stimulus. Subjects reported whether they heard the consonant
/b/ or another consonant; fewer /b/ responses when the audio and
visual stimuli were mismatched indicated higher levels of visual
in£uence on speech perception (McGurk eĦect). Active brain
regions during presentation of the incongruent stimuli included
the superior temporal and inferior frontal gyrus, as well as extrastriate,
premotor and posterior parietal cortex. A regression analysis
related the strength of the McGurk eĦect to levels of brain
activation. Paradoxically, higher numbers of /b/ responses were
positively correlated with activation in the left occipito-temporal
junction, an area often associated with processing visual motion.
This activation suggests that auditory informationmodulatesvisual
processing to aĦect perception.
Download: PDF (149k) JonesCallanNeuroReport03.pdf
revised Feb 12/08
View all Jeffery A. Jones documents