Avatars make the Internet sign to deaf people

Aug 29, 2014
Computer scientists from Saarbrücken in Germany have developed an approach where animated online characters display content in sign language. They collaborate with Peter Schaar (picture) who is deaf and is a lecturer for sign language. Credit: AG Heloir

It is challenging for deaf people to learn a sound-based language, since they are physically not able to hear those sounds. Hence, most of them struggle with written language as well as with text reading and comprehension. Therefore, most website content remains inaccessible for them. Computer scientists from Saarbrücken, Germany, want to change the situation by means of a method they developed: animated online characters display content in sign language. In the long term, deaf people would be able to use the technique to communicate on online platforms via sign language. To realize the technique, users would only need readily available devices.

Sign language was first acknowledged as a separate language in the sixties, which is quite recent. Similar to , it evolved from different cultural backgrounds. Every country has its own with various dialects, which are based on different rules than the spoken language. For the deaf, sign language is their native language. Therefore, it is not easy for them to learn spoken language, which is why they may struggle with text reading and comprehension even after their graduation. Although several websites provide video clips in which sign language interpreters translate the text, much Internet content remains cryptic for the deaf community. To inform quickly in cases where there is no interpreter on hand, researchers are working on a novel approach to provide content. Their idea: avatars. These animated characters could be used in the context of announcements at train stations, or on websites.

"We have already gained initial experiences with avatars", explains Alexis Heloir, who is the leader of the research group "Sign Language Synthesis and Interaction" at the Cluster of Excellence "Multimodal Computing and Interaction" and also a researcher at the German Center for Artificial Intelligence. "If we try to animate them like human beings, deaf people have issues with understanding the avatars." The researcher assumes that this is caused by the greater variety of emotional expressions of humans compared to avatars. To deal with that problem, Alexis Heloir and Fabrizio Nunnari create avatars that make more accentuated movements. The researchers are closely cooperating with Peter Schaar, who is deaf and is a lecturer for German sign language at the Saarland University Language Center and the College of Engineering and Commerce in Saarbrücken.

"Our method should be inexpensive and easy to use so that every member of the deaf community will be able to use it", says Fabrizio Nunnari. To capture the motions of deaf people, the scientists make use of affordable cameras and sensors that are typically used by teenagers for computer games. A computing method transfers the movements of the entire body onto the avatar. In the long term, the researchers want to create a collection of short sign language sequences that can be used by the deaf to interact on the web.

Explore further: Hand gestures improve learning in both signers and speakers

add to favorites email to friend print save as pdf

Related Stories

Early exposure to language for deaf children

Jun 05, 2012

(Medical Xpress) -- Most agree that the earlier you expose a child to a language, the easier it is for that child to pick it up. The same rules apply for deaf children.

Recommended for you

Forging a photo is easy, but how do you spot a fake?

Nov 21, 2014

Faking photographs is not a new phenomenon. The Cottingley Fairies seemed convincing to some in 1917, just as the images recently broadcast on Russian television, purporting to be satellite images showin ...

Algorithm, not live committee, performs author ranking

Nov 21, 2014

Thousands of authors' works enter the public domain each year, but only a small number of them end up being widely available. So how to choose the ones taking center-stage? And how well can a machine-learning ...

Professor proposes alternative to 'Turing Test'

Nov 19, 2014

(Phys.org) —A Georgia Tech professor is offering an alternative to the celebrated "Turing Test" to determine whether a machine or computer program exhibits human-level intelligence. The Turing Test - originally ...

Image descriptions from computers show gains

Nov 18, 2014

"Man in black shirt is playing guitar." "Man in blue wetsuit is surfing on wave." "Black and white dog jumps over bar." The picture captions were not written by humans but through software capable of accurately ...

Converting data into knowledge

Nov 17, 2014

When a movie-streaming service recommends a new film you might like, sometimes that recommendation becomes a new favorite; other times, the computer's suggestion really misses the mark. Yisong Yue, assistant ...

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.