Findings could lead to improved lip-reading training for the deaf and hard-of-hearing

Sep 10, 2009
A new study by the University of East Anglia suggests computers are now better at lip-reading than humans. Credit: Sarah Hilder/UEA

A new study by the University of East Anglia suggests computers are now better at lip-reading than humans.

The peer-reviewed findings will be presented for the first time at the eighth International Conference on Auditory-Visual Speech Processing (AVSP) 2009, held at the University of East Anglia from September 10-13.

A research team from the School of Computing Sciences at UEA compared the performance of a machine-based lip-reading system with that of 19 human lip-readers. They found that the automated system significantly outperformed the human lip-readers - scoring a recognition rate of 80 per cent, compared with only 32 per cent for human viewers on the same task.

Furthermore, they found that machines are able to exploit very simplistic features that represent only the shape of the face, whereas human lip-readers require full video of people speaking.

The study also showed that rather than the traditional approach to lip-reading training, in which viewers are taught to spot key lip-shapes from static (often drawn) images, the dynamics and the full appearance of speech gestures are very important.

Using a new video-based training system, viewers with very limited training significantly improved their ability to lip-read monosyllabic words, which in itself is a very difficult task. It is hoped this research might lead to novel methods of lip-reading training for the deaf and hard of hearing.

"This pilot study is the first time an automated lip-reading system has been benchmarked against human lip-readers and the results are perhaps surprising," said the study's lead author Sarah Hilder.

"With just four hours of training it helped them improve their lip-reading skills markedly. We hope this research will represent a real technological advance for the deaf community."

Agnes Hoctor, campaigns manager at the RNID, said: "This research confirms how difficult the vital skill of lip-reading is to learn and why RNID is campaigning for people who are deaf or hard of hearing to have improved access to classes. We would welcome the development of video-based or online training resources to supplement the teaching of lip-reading. Hearing loss affects 55 per cent of people over 60 so, with the ageing population, demand to learn lip-reading is only going to increase."

The AVSP conference is being held in the UK for the first time since its inception in 1998. The University of East Anglia will host cutting edge researchers including psychologists, engineers, scientists and linguists from as far afield as Australia, Canada and Japan.

Source: University of East Anglia

Explore further: The 'fifth taste,' umami, could be beneficial for health

add to favorites email to friend print save as pdf

Related Stories

Lip-read me now, hear me better later

Apr 12, 2007

Experience hearing a person's voice allows us to more easily hear what they are saying. Now research by UC Riverside psychology Professor Lawrence D. Rosenblum and graduate students Rachel M. Miller and Kauyumari Sanchez ...

Recommended for you

Is head CT overused in emergency departments?

4 hours ago

Less than 7.1% of patients presenting to the emergency department with dizziness and 6.4% complaining of syncope or near-syncope benefited from head CT say researchers at Kaiser Foundation Hospital in Honolulu. The use of ...

Medical radiation may be reduced to one-sixth

7 hours ago

One of this century's most significant mathematical discoveries may reduce the number of measuring points to one-sixth of the present level. This means reduced exposure to radiation and faster medical imaging ...

The 'fifth taste,' umami, could be beneficial for health

20 hours ago

The umami taste could have an important and beneficial role in health, according to research published in the open access journal Flavour. The journal's special series of articles 'The Science of Taste' also finds that ' ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

RayCherry
not rated yet Sep 10, 2009
Finally. :-)

Soon we will have a pair of glasses with stereo cameras with an iPod like device for image processing and builtin head phones, permitting the hard-of-hearing to receive a real assistance - filtering the desired voice from the background.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.