Jeffery A. Jones
Visual prosody and speech intelligibility: Head movement improves auditory speech perception.
Psychological Science, 15, 133-137.
Munhall, K. G., Jones, J. A., Callan, D. E., Kuratate, T. & Vatikiotis-Bateson, E.
published: 2004 | Research publication | Jones Lab
People naturally move their heads when they speak,
and our study shows that this rhythmic head motion conveys
linguistic information. Three-dimensional head and face
motion and the acoustics of a talker producing Japanese sentences
were recorded and analyzed. The head movement correlated
strongly with the pitch (fundamental frequency) and
amplitude of the talkerís voice. In a perception study, Japanese
subjects viewed realistic talking-head animations based on these
movement recordings in a speech-in-noise task. The animations
allowed the head motion to be manipulated without changing
other characteristics of the visual or acoustic speech. Subjects
correctly identified more syllables when natural head motion
was present in the animation than when it was eliminated
or distorted. These results suggest that nonverbal gestures
such as head movements play a more direct role in the perception
of speech than previously known.
Download: PDF (142k) †† Munhall_Psyc.pdf
revised Feb 12/08
View all Jeffery A. Jones documents