Findings could lead to improved lip-reading training for the deaf and hard-of-hearing

Sep 10, 2009
A new study by the University of East Anglia suggests computers are now better at lip-reading than humans. Credit: Sarah Hilder/UEA

A new study by the University of East Anglia suggests computers are now better at lip-reading than humans.

The peer-reviewed findings will be presented for the first time at the eighth International Conference on Auditory-Visual Speech Processing (AVSP) 2009, held at the University of East Anglia from September 10-13.

A research team from the School of Computing Sciences at UEA compared the performance of a machine-based lip-reading system with that of 19 human lip-readers. They found that the automated system significantly outperformed the human lip-readers - scoring a recognition rate of 80 per cent, compared with only 32 per cent for human viewers on the same task.

Furthermore, they found that machines are able to exploit very simplistic features that represent only the shape of the face, whereas human lip-readers require full video of people speaking.

The study also showed that rather than the traditional approach to lip-reading training, in which viewers are taught to spot key lip-shapes from static (often drawn) images, the dynamics and the full appearance of speech gestures are very important.

Using a new video-based training system, viewers with very limited training significantly improved their ability to lip-read monosyllabic words, which in itself is a very difficult task. It is hoped this research might lead to novel methods of lip-reading training for the deaf and hard of hearing.

"This pilot study is the first time an automated lip-reading system has been benchmarked against human lip-readers and the results are perhaps surprising," said the study's lead author Sarah Hilder.

"With just four hours of training it helped them improve their lip-reading skills markedly. We hope this research will represent a real technological advance for the deaf community."

Agnes Hoctor, campaigns manager at the RNID, said: "This research confirms how difficult the vital skill of lip-reading is to learn and why RNID is campaigning for people who are deaf or hard of hearing to have improved access to classes. We would welcome the development of video-based or online training resources to supplement the teaching of lip-reading. Hearing loss affects 55 per cent of people over 60 so, with the ageing population, demand to learn lip-reading is only going to increase."

The AVSP conference is being held in the UK for the first time since its inception in 1998. The University of East Anglia will host cutting edge researchers including psychologists, engineers, scientists and linguists from as far afield as Australia, Canada and Japan.

Source: University of East Anglia

Explore further: Key element of CPR missing from guidelines

add to favorites email to friend print save as pdf

Related Stories

Lip-read me now, hear me better later

Apr 12, 2007

Experience hearing a person's voice allows us to more easily hear what they are saying. Now research by UC Riverside psychology Professor Lawrence D. Rosenblum and graduate students Rachel M. Miller and Kauyumari Sanchez ...

Recommended for you

Exploring 3-D printing to make organs for transplants

8 hours ago

Printing whole new organs for transplants sounds like something out of a sci-fi movie, but the real-life budding technology could one day make actual kidneys, livers, hearts and other organs for patients ...

High frequency of potential entrapment gaps in hospital beds

9 hours ago

A survey of beds within a large teaching hospital in Ireland has shown than many of them did not comply with dimensional standards put in place to minimise the risk of entrapment. The report, published online in the journal ...

Key element of CPR missing from guidelines

Jul 29, 2014

Removing the head tilt/chin lift component of rescue breaths from the latest cardiopulmonary resuscitation (CPR) guidelines could be a mistake, according to Queen's University professor Anthony Ho.

Burnout impacts transplant surgeons (w/ Video)

Jul 28, 2014

Despite saving thousands of lives yearly, nearly half of organ transplant surgeons report a low sense of personal accomplishment and 40% feel emotionally exhausted, according to a new national study on transplant surgeon ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

RayCherry
not rated yet Sep 10, 2009
Finally. :-)

Soon we will have a pair of glasses with stereo cameras with an iPod like device for image processing and builtin head phones, permitting the hard-of-hearing to receive a real assistance - filtering the desired voice from the background.