New sofware brings down communication barriers for the hard of hearing
The telephone is taken for granted by most, but for the hard of hearing it can be a real obstacle to communication. Overcoming the difficulties faced by people with hearing problems is Synface, a software program that generates an animated face synchronised with speech.
Installed on a regular computer and using a standard phone line, the software allows the hard of hearing to lip read while hearing the person they are talking to at the same time, a technique that is showing enormous potential in bringing down communications barriers for the hard of hearing.
“The Synface system is an important step in helping the people with hearing difficulties communicate, it assists them in using the phone through providing not only audio communication but also visual,” explains Inger Karlsson at KTH in Sweden, which is coordinating the IST programme-funded project.
Due to end in December, the Synface project has developed a unique system, with the project partners taking a new approach to combining computer-generated synthetic faces and sound-recognition technologies. It offers considerable advantages over other systems, and important benefits to users, as recent tests indicate.
Positive feedback from trials
Preliminary results from ongoing trials in the United Kingdom show that 84 per cent of users found the system helped them recognise words, while 74 per cent said it made telephone conversations more effective and assisted them in conversing normally. Trials in Sweden and The Netherlands are expected to produce similar results.
“Most of the trial users were very positive about the system, although some got more out of it than others as its efficiency depends on how well a user can hear and also on how well they can lip read,” Karlsson says. “Even so, it has distinct advantages over other techniques.”
Advantages over other techniques
The two most common methods to assist telephone communication employed to date are video phones and text phones, although these can be impractical. Video phones, for example, require units on both ends of the line and large bandwidth in order for a hard of hearing person to be able to lip read from the face of the person they are talking to. Text phones require typing, making normal conversation slow or impossible.
Synface on the other hand makes talking on the phone almost as natural for people with hearing problems as holding a face-to-face conversation. The software contains a speech recogniser that converts the audio signal into the lip movements of the animated face on a PC screen. Synchronisation between the audio and visual communication is achieved through a 200 millisecond delay to allow processing.
“The system doesn’t recognise words, it recognises sounds,” Karlsson notes. “When we speak we know what we are going to say next and the word we say previously influences the pronunciation of the following word. Therefore the system has a delay allowing it to recognise what sound is coming next before the audio and visual communication is passed onto the user.”
By recognising sounds as opposed to words, Synface drastically reduces the time in which it takes the lip movements to be generated - which amounted to as much as half a second in other systems – while making it easier to employ in multiple languages. As Synface does not contain a lexicon, it only needs to be programmed with sounds that are specific to different languages and needs very few changes if it is being adapted between similar languages such as Swedish and Norwegian or Italian and Spanish. So far the prototype has been developed for use in English, Swedish and Dutch, with demonstrations in Italian and Finnish currently being prepared prior to a commercial variant being launched within the next two years.
A potentially huge market
The European market for such a system is potentially huge. Although no precise data exists on the number of people with some form of hearing impairment, some studies put it as high as 80 million across the continent. Karlsson notes, for example, that out of Sweden’s population of nine million, “an estimated 800,000 people have hearing problems, and of these 350,000 need hearing aids or other assistance.”
Even though Synface is not suitable for the totally deaf or people with minimal hearing difficulties who could find the face distracting, Karlsson indicates that a large number of people could make use of the system, especially as health problems related to hearing are likely to increase over the coming years.
“With an aging population in Europe the number of cases of hearing impairment are on the rise, while some studies are predicting that people will start suffering hearing problems earlier in life because of the high levels of noise young people are exposed to today,” the project coordinator says, noting that the hard of hearing may not be the only beneficiaries of the system.
“At some point we’ve all been hard of hearing, such as when we’re in a noisy train station,” Karlsson says. “The animated face could be used in combination with audio announcements to make them easier to understand.”
There is also the potential for Synface to be employed on laptop computers, PDAs and mobile phones – making communication easier for everyone, anywhere and at any time.
Source: ISTresults
