Findings could lead to improved lip-reading training for the deaf and hard-of-hearing

Sep 10, 2009
A new study by the University of East Anglia suggests computers are now better at lip-reading than humans. Credit: Sarah Hilder/UEA

A new study by the University of East Anglia suggests computers are now better at lip-reading than humans.

The peer-reviewed findings will be presented for the first time at the eighth International Conference on Auditory-Visual Speech Processing (AVSP) 2009, held at the University of East Anglia from September 10-13.

A research team from the School of Computing Sciences at UEA compared the performance of a machine-based lip-reading system with that of 19 human lip-readers. They found that the automated system significantly outperformed the human lip-readers - scoring a recognition rate of 80 per cent, compared with only 32 per cent for human viewers on the same task.

Furthermore, they found that machines are able to exploit very simplistic features that represent only the shape of the face, whereas human lip-readers require full video of people speaking.

The study also showed that rather than the traditional approach to lip-reading training, in which viewers are taught to spot key lip-shapes from static (often drawn) images, the dynamics and the full appearance of speech gestures are very important.

Using a new video-based training system, viewers with very limited training significantly improved their ability to lip-read monosyllabic words, which in itself is a very difficult task. It is hoped this research might lead to novel methods of lip-reading training for the deaf and hard of hearing.

"This pilot study is the first time an automated lip-reading system has been benchmarked against human lip-readers and the results are perhaps surprising," said the study's lead author Sarah Hilder.

"With just four hours of training it helped them improve their lip-reading skills markedly. We hope this research will represent a real technological advance for the deaf community."

Agnes Hoctor, campaigns manager at the RNID, said: "This research confirms how difficult the vital skill of lip-reading is to learn and why RNID is campaigning for people who are deaf or hard of hearing to have improved access to classes. We would welcome the development of video-based or online training resources to supplement the teaching of lip-reading. Hearing loss affects 55 per cent of people over 60 so, with the ageing population, demand to learn lip-reading is only going to increase."

The AVSP conference is being held in the UK for the first time since its inception in 1998. The University of East Anglia will host cutting edge researchers including psychologists, engineers, scientists and linguists from as far afield as Australia, Canada and Japan.

Source: University of East Anglia

Explore further: Continued reliance on Windows XP in physician practices may threaten data security

add to favorites email to friend print save as pdf

Related Stories

Lip-read me now, hear me better later

Apr 12, 2007

Experience hearing a person's voice allows us to more easily hear what they are saying. Now research by UC Riverside psychology Professor Lawrence D. Rosenblum and graduate students Rachel M. Miller and Kauyumari Sanchez ...

Recommended for you

Obese British man in court fight for surgery

Jul 11, 2011

A British man weighing 22 stone (139 kilograms, 306 pounds) launched a court appeal Monday against a decision to refuse him state-funded obesity surgery because he is not fat enough.

2008 crisis spurred rise in suicides in Europe

Jul 08, 2011

The financial crisis that began to hit Europe in mid-2008 reversed a steady, years-long fall in suicides among people of working age, according to a letter published on Friday by The Lancet.

New food labels dished up to keep Europe healthy

Jul 06, 2011

A groundbreaking deal on compulsory new food labels Wednesday is set to give Europeans clear information on the nutritional and energy content of products, as well as country of origin.

Overweight men have poorer sperm count

Jul 04, 2011

Overweight or obese men, like their female counterparts, have a lower chance of becoming a parent, according to a comparison of sperm quality presented at a European fertility meeting Monday.

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

RayCherry
not rated yet Sep 10, 2009
Finally. :-)

Soon we will have a pair of glasses with stereo cameras with an iPod like device for image processing and builtin head phones, permitting the hard-of-hearing to receive a real assistance - filtering the desired voice from the background.

More news stories

Unraveling the 'black ribbon' around lung cancer

It's not uncommon these days to find a colored ribbon representing a disease. A pink ribbon is well known to signify breast cancer. But what color ribbon does one think of with lung cancer?

Classifying cognitive styles across disciplines

Educators have tried to boost learning by focusing on differences in learning styles. Management consultants tout the impact that different decision-making styles have on productivity. Various fields have ...

Tiny power plants hold promise for nuclear energy

Small underground nuclear power plants that could be cheaper to build than their behemoth counterparts may herald the future for an energy industry under intense scrutiny since the Fukushima disaster, the ...

Hand out money with my mobile? I think I'm ready

A service is soon to launch in the UK that will enable us to transfer money to other people using just their name and mobile number. Paym is being hailed as a revolution in banking because you can pay peopl ...