Visual system interprets sign languages

Jun 02, 2010

Spanish sign language is used by over 100,000 people with hearing impairments and is made up of hundreds of signs. CVC-UAB researchers Sergio Escalera, Petia Radeva and Jordi Vitria selected over twenty of these signs to develop a new visual interpretation system which allows deaf people to carry out consultations in the language they commonly use.

Signs can vary slightly depending on each user. Project researchers took this into account during the trials carried out with different people to help the system "become familiarised" with this variability. The signs recognised by the system were programmed to allow deaf people to maintain a basic conversation, including asking for help or directions. "For them it is a non artificial way of communicating and at the same time they can engage with people who do not speak sign language since the system translates the symbols into words in real time," Sergio Escalera said.

The hardware includes a which records image sequences when it detects the presence of a user wanting to make a consultation. A and automatic learning system detects face, hand and arm movements, as well as any screen scrolling, and incorporates these into a which identifies each movement with the word associated with the sign.

One of the aspects worth highlighting is the ability to adapt the system to any other , since the methodology used is general. The system would only need to be reprogrammed with the signs used in that specific language. The amount of signs the system can recognise is also scalable, although researchers do admit that new data will increase the difficulty in differentiating them.

Applications such as the one developed by CVC-UAB researchers require extreme precision in the identification phase and are very difficult to configure given that the surroundings in which they will be used include changes in light and shadow, different physiognomies and speeds at which the signs are formed.

Other similar projects have been developed in the past. However, most of them failed or were not reliable enough because of the high complexity of variabilities in uncontrolled surroundings. For this project to succeed it was necessary to establish a fixed point in which individuals formed the signs and avoid having different focus points when recording.

The system was recently presented as a prototype in the final phase of a European project and researchers are already working on new project phases, such as using two cameras with the aim of recognising even more complex signs and complementing information with facial characteristics. To carry this out researchers worked in close collaboration with several members of the Catalan Federation of , FECOSA.

Explore further: Sweeping air devices for greener planes

Provided by Universitat Autonoma de Barcelona

4.5 /5 (2 votes)
add to favorites email to friend print save as pdf

Related Stories

Sign language cell phone service created

Mar 06, 2007

The world's first sign language dictionary available from a mobile phone has been launched by the University of Bristol's Centre for Deaf Studies.

Deaf children use hands to invent own way of communicating

Feb 15, 2009

Deaf children are able to develop a language-like gesture system by making up hand signs and using homemade systems to increase their communication as they grow, just as children with conventional spoken language, research ...

'Deaf by God' tried in Old Bailey records

May 05, 2008

Deaf people on trial were granted the right to an interpreter as early as 1725, according to Old Bailey records examined by UCL (University College London) scientists. The use of family and friends to interpret court proceedings ...

Sign language puzzle solved

Dec 15, 2009

(PhysOrg.com) -- Scientists have known for 40 years that even though it takes longer to use sign language to sign individual words, sentences can be signed, on average, in the same time it takes to say them, ...

Researchers create cell phones for sign language

Dec 02, 2009

(PhysOrg.com) -- Cornell researchers and colleagues have created cell phones that allow deaf people to communicate in sign language, the same way hearing people use phones to talk.

Recommended for you

Sweeping air devices for greener planes

11 hours ago

The large amount of jet fuel required to fly an airplane from point A to point B can have negative impacts on the environment and—as higher fuel costs contribute to rising ticket prices—a traveler's wallet. ...

User comments : 0