Purdue phoneme project creates new haptic communications future

June 18, 2018 by Brian L. Huchel, Purdue University
Postdoctoral student Yang Jiao communicates words to Jaeong Jung, an undergraduate student, using phoneme signals transmitted to the haptic device on his forearm. Credit: Purdue University photo/Brian Huchel

Communication could step beyond reading a cellular phone screen with a new technique by Purdue College of Engineering researchers to learn and read messages through a person's sense of touch.

Hong Tan, a professor of electrical and computer engineering, was the lead professor on the haptic research that developed a method to receive messages by learning to interpret signals such as a buzzing sensation and others through the skin on the forearm.

The were presented Friday (June 15) at the Proceedings of EuroHaptics 2018 conference in Pisa, Italy.

 The yearlong project was done with researchers from the Massachusetts Institute of Technology and Facebook Inc.

Tan, founder and director of Purdue's Haptic Interface Research Laboratory, said while the research lends itself to being used by hearing-impaired and visually impaired users, the study is also being looked at on general terms for any number of possible uses.

"We are collaborating with Facebook through the company's Sponsored Academic Research Agreement. Facebook is interested in developing new platforms for communication and the haptic research we are doing has been promising," she said.

"I'm excited about this … imagine a future where you're able to wear a sleeve that discreetly sends messages to you – through your skin – in times when it may be inconvenient to look at a text message," Tan said. "I'm really hoping this takes off as a general idea for a new way to communicate.

"When that happens, the hearing-impaired, the visually-impaired, everyone can benefit."

In the study, subjects used a material cuff encircling the forearm from the wrist to below the elbow. The instrument, wrapped around the test subject's non-dominant arm, featured 24 tactors that, when stimulated, emitted a vibration against the skin, changing quality and position in the process.

Tan said the 39 phonemes (units of sound in a language that distinguish one word from another) in the English language were mapped using signals from specific tactors. The sounds of consonants such as K, P and T were stationary sensations on different areas of the arm while vowels were indicated by stimulations that moved up, down or around the forearm.

"We used anything that can help you establish the mapping and recognize and memorize it," Tan said. "This is based on a better understanding of how to transmit information through the sense of touch in a more intuitive and effective manner."

Twelve subjects learned haptic symbols through the phoneme-based method at a schedule of 100 words in 100 minutes for the research, while 12 others learned using a word-based system with the haptic signals on their arm.

Research results showed the phonemes worked better than a word-based approach by providing a more consistent path for user learning in a shorter period of time. Performance levels varied greatly among the test subjects for each method, but with the phoneme-based approach at least half could perform at 80 percent accuracy while two subjects reached 90 percent accuracy.

Tan said using phonemes was more efficient compared to letters, noting there are less phonemes in a word compared with the number of letters.

She said the reasoning behind the project is the thought that there are many ways to encode speech into feelings. The project goal is to demonstrate one way that actually works.

"For this research, the learning progress is one of the key things," Tan said. "With this, not only do we have a system that works, but we're able to train people within hours rather than months or even years."

For the study, a concentrated, intense training regimen was developed where the worked for about 10 minutes a day. They were then tested on their progress.

"It is more efficient than if they sit here for three hours to study," Tan said. "You can't keep good concentration for that long."

"People who are hearing-impaired may be motivated to spend additional time to training themselves, but the general public probably doesn't have the patience," she added.

Explore further: Why deaf people can have accents, too

More information: A Comparative Study of Phoneme- and Word-Based Learning of English Words Presented to the Skin, Proceedings of EuroHaptics 2018 conference.

Related Stories

Why deaf people can have accents, too

March 22, 2017

Most people have probably encountered someone who appears to use lip-reading to overcome a hearing difficulty. But it is not as simple as that. Speech is "bimodal", in that we use both sounds and facial movements and gestures ...

Recommended for you

Technology near for real-time TV political fact checks

January 18, 2019

A Duke University team expects to have a product available for election year that will allow television networks to offer real-time fact checks onscreen when a politician makes a questionable claim during a speech or debate.

Privacy becomes a selling point at tech show

January 7, 2019

Apple is not among the exhibitors at the 2019 Consumer Electronics Show, but that didn't prevent the iPhone maker from sending a message to attendees on a large billboard.

China's Huawei unveils chip for global big data market

January 7, 2019

Huawei Technologies Ltd. showed off a new processor chip for data centers and cloud computing Monday, expanding into new and growing markets despite Western warnings the company might be a security risk.

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

rrwillsj
1 / 5 (2) Jun 19, 2018
This research has additional relevancy for the Space Program. Between unexpected radiation events and high oxygen content in spacecraft? Planning deep space manned operations needs to have a back-up plan for if the crew are sight-impaired or blinded.

Trained in the procedures outlined in this article. Using controls, instruments and accessories with tactile communication. Would be a foresightful investment to insure that the mission can be completed and the crew return alive.

What about direct brain-to-machine link? Probably useful if that unexpected radiation effect did not damage the link at either end. Or an oxygen fire does not leave damage and contaminants all throughout an enclosed environment.

KISS, haptic systems would be one more, very basic, very rugged back-up to the back-up.

A hundred million miles fro Earth, no AutoClub towtruck is showing up to haul your sorry ass home!

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.