Language learning: Researchers use video games to crack the speech code

May 18, 2011

When we speak, our enunciation and pronunciation of words and syllables fluctuates and varies from person to person. Given this, how do infants decode all of the spoken sounds they hear to learn words and meanings?

To replicate the challenges of as an infant, Carnegie Mellon University's Lori Holt and Sung-Joo Lim and Stockholm University's Francisco Lacerda used video game training with a mock "alien" language. They discovered that listeners quickly recognize word-like units. They will present their findings at the Acoustical Society of America's annual meeting May 23-27 in Seattle.

To uncover how spoken sounds are decoded, the research team designed a video game narrated in deliberately distorted speech. The soundtrack, unintelligible in any language, was the only source of instruction for 77 adult players in the study. With just two hours of play, the participants could reliably extract world-length sound categories from continuous alien sounds and apply that learning to advance through the game.

"Traditionally, when we study adult learning in the lab, it's nothing like how infants learn language," said Holt, professor of psychology at CMU and a specialist in auditory . "This video game models for adults the challenge language learning poses to infants. This presents the opportunity to study learning in ways that are just not feasible with infants."

Lacerda, professor of and an expert in , agrees that the use of a video game is a promising new way to explore language learning. "This is a wonderful opportunity to approximate the task facing infants by creating a setting where adults are forced to infer what the meaning of different sound elements might be, and to do it in a functional way."

Understanding how language is learned has broad implications from treating dyslexia — which causes difficulty identifying functional sound units — to improving second . Lim, a graduate student in psychology at CMU and lead author of the study, has used the game to help adults learn English. "Native speakers of Japanese can use this type of training to learn English consonants they have difficulty distinguishing," she said.

Holt, director of CMU's Speech Perception and Learning Laboratory, investigates how the brain interprets sound in order to provide a model of auditory categorization to solve problems related to speech perception and communication disorders. Holt and her colleagues are interested in taking this current study further to determine how the video game and its alien soundtrack engage different areas of the brain to produce rapid and robust learning. Their next step is to investigate this by observing players with functional magnetic resonance imaging (fMRI) to view their real-time brain reactions to the .

Explore further: Newlyweds, be careful what you wish for

Related Stories

Language learning influenced by genes

Feb 11, 2011

Researchers at the University of Edinburgh have found a gene - called ROBO1 - linked to the mechanism in the brain that helps infants develop speech.

Adults with dyslexia have problems with non-speech sounds too

Jun 01, 2010

(PhysOrg.com) -- Dyslexia is usually associated with persistent reading, spelling, and sometimes speech difficulties that are hard to overcome. One theory proposed to explain the condition is that people with dyslexia suffer ...

New research demonstrates language learners' creativity

Mar 15, 2011

New research published in Language, the journal of the Linguistic Society of America (LSA) firmly establishes that language learning goes well beyond simple imitation, and in fact that language learners are quite creative and re ...

Recommended for you

Newlyweds, be careful what you wish for

16 hours ago

A statistical analysis of the gift "fulfillments" at several hundred online wedding gift registries suggests that wedding guests are caught between a rock and a hard place when it comes to buying an appropriate gift for the ...

Can new understanding avert tragedy?

19 hours ago

As a boy growing up in Syracuse, NY, Sol Hsiang ran an experiment for a school project testing whether plants grow better sprinkled with water vs orange juice. Today, 20 years later, he applies complex statistical ...

Creative activities outside work can improve job performance

Apr 16, 2014

Employees who pursue creative activities outside of work may find that these activities boost their performance on the job, according to a new study by San Francisco State University organizational psychologist Kevin Eschleman ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

hush1
not rated yet May 18, 2011
Metaphor

How long have I waited for someone to see the light?
How long have I waited for someone to hear the sound?

Kudos all around. Congratulations.

Child's play to complete a robust theory of language learning.
Long life convictions, vetted, vindicated, black on white.

I expect to see you all in Sweden. If that is of value to you.
Key research references will aid you to home plate.

Long life work, convictions, vetted, vindicated, black on white, for all to read. There is no better vindication.

The next step you are taking is not direct. I am not going to intervene. You will still reach home plate.


More news stories

Newlyweds, be careful what you wish for

A statistical analysis of the gift "fulfillments" at several hundred online wedding gift registries suggests that wedding guests are caught between a rock and a hard place when it comes to buying an appropriate gift for the ...

Can new understanding avert tragedy?

As a boy growing up in Syracuse, NY, Sol Hsiang ran an experiment for a school project testing whether plants grow better sprinkled with water vs orange juice. Today, 20 years later, he applies complex statistical ...

Roman dig 'transforms understanding' of ancient port

(Phys.org) —Researchers from the universities of Cambridge and Southampton have discovered a new section of the boundary wall of the ancient Roman port of Ostia, proving the city was much larger than previously ...

Crowd-sourcing Britain's Bronze Age

A new joint project by the British Museum and the UCL Institute of Archaeology is seeking online contributions from members of the public to enhance a major British Bronze Age archive and artefact collection.

Better thermal-imaging lens from waste sulfur

Sulfur left over from refining fossil fuels can be transformed into cheap, lightweight, plastic lenses for infrared devices, including night-vision goggles, a University of Arizona-led international team ...

Hackathon team's GoogolPlex gives Siri extra powers

(Phys.org) —Four freshmen at the University of Pennsylvania have taken Apple's personal assistant Siri to behave as a graduate-level executive assistant which, when asked, is capable of adjusting the temperature ...