Read my lips: Using multiple senses in speech perception (Video)

Feb 11, 2009

When someone speaks to you, do you see what they are saying? We tend to think of speech as being something we hear, but recent studies suggest that we use a variety of senses for speech perception - that the brain treats speech as something we hear, see and even feel. In a new report in Current Directions in Psychological Science, a journal of the Association for Psychological Science, psychologist Lawrence Rosenblum describes research examining how our different senses blend together to help us perceive speech.

We receive a lot of our speech information via visual cues, such as lip-reading, and this type of visual speech occurs throughout all cultures. And it is not just information from lips- when someone is speaking to us, we will also note movements of the teeth, tongue and other non-mouth facial features. It's likely that human speech perception has evolved to integrate many senses together. Put in another way, speech is not meant to be just heard, but also to be seen.

This video is not supported by your browser at this time.

The McGurk Effect is a well-characterized example of the integration between what we see and what we hear when someone is speaking to us. This phenomenon occurs when a sound (such as a syllable or word) is dubbed with a video showing a face making a different sound. For example, the audio may be playing "ba," while the face looks as though it is saying "va." When confronted with this, we will usually hear "va" or a combination of the two sounds, such as "da." Interestingly, when study participants are aware of the dubbing or told to concentrate only on the audio, the McGurk Effect still occurs. Rosenblum suggests that this is evidence that once senses are integrated together, it is not possible to separate them.

Recent studies indicate that this integration occurs very early in the speech process, even before phonemes (the basic units of speech) are established. Rosenblum suggests that physical movement of speech (that is, our mouths and lips moving) create acoustic and visual signals which have a similar form. He argues that as far as the speech brain is concerned, the auditory and visual information are never really separate. This could explain why we integrate speech so readily and in such a way that the audio and visual speech signals become indistinguishable from one another.

Rosenblum concludes that visual-speech research has a number of clinical implications, especially in the areas of autism, brain injury and schizophrenia and that "rehabilitation programs in each of these domains have incorporated visual-speech stimuli."

Source: Association for Psychological Science

Explore further: Coping skills training in rheumaotoid arthritis research recognized by BSBS of MI

Related Stories

Economic mobility is an illusory American dream

Feb 04, 2015

Americans blithely buy in to a socio-economic system of increasingly vast financial inequity because we believe – despite evidence to the contrary – that everyone still has the opportunity to succeed, new studies by two ...

New avatars capable of laughing

Apr 04, 2014

Today's computer-based avatars lack one of our most deeply rooted human characteristics: laughter. Computer scientists have now teamed up with psychologists to give avatars the ability to laugh.

'The King's Speech': good drama - but accurate science?

Feb 21, 2011

(PhysOrg.com) -- "The King's Speech" is a compelling enough story to merit 12 Oscar nominations. (We’ll find out how compelling when the Academy Awards are announced Feb. 27). However, as contentions surface about the ...

Recommended for you

Demi Lovato gets vocal about mental illness

3 hours ago

(HealthDay)—Demi Lovato huddled in the back of her tour bus, eyes wet with tears as she watched a horde of fans streaming into the venue where she was about to play.

Acquiring 'perfect' pitch may be possible for some adults

3 hours ago

If you're a musician, this sounds too good to be true: University of Chicago psychologists have been able to train some adults to develop the prized musical ability of absolute pitch, and the training's effects ...

How men and women see each other when online dating

5 hours ago

In the world of online dating, nothing is as it seems. But that doesn't stop many of us from leaping to the wrong conclusions about people. A recent paper presented at the Annual Conference of the International ...

We trust kids to know what gender they are

5 hours ago

I will start by asking two questions: at what age did you know your gender, and do you think someone else had to tell you what it was? I'm director of mental health at a leading gender clinic in the US. Our clinic is a half-decade old – and in that short pe ...

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.