Beyond tapping and sliding

Aug 06, 2014
Beyond tapping and sliding
An electrostatic haptics prototype on which the gray area feels rough, like sandpaper, and the dotted line has a notched texture for scrolling.

"The way we design computers today," Microsoft researcher Hong Tan says, "it would seem that people only use their eyes."

Sure, we tap on our device screens, slide our fingertips across the glass, and type on on-screen keyboards. Sometimes, we give voice commands and listen back. Our phones vibrate when a text arrives, and we feel a rumble from a joystick when we play a video game.

But still, it's almost entirely about the eyes.

Tan believes we have barely begun to engage our other senses, particularly the one in which she specializes: touch. Her field of expertise is computer haptics—developing hardware and software to add tactile sensations to the computing experience.

Imagine experiencing a clicking sensation when pressing an on-screen button, sensing the weight of folders when dragging and dropping, and perhaps even feeling the texture of a sweater for sale online.

"With sight alone, most people are perfectly fine interacting with computing devices today," Tan says, "but how much more efficiently, how much more enjoyably, can we interact with computers? How much more accessible can we make them? We won't know until this becomes taken for granted."

Based at Microsoft Research Asia, Tan recently introduced her work and the field of computer haptics to her largest audience yet via a popular technology column on Sina.com, the largest Chinese-language web portal.

Tan begins with the basic assumption that we are multisensory beings—a perspective she absorbed at the start of her research career. As a graduate student in electrical engineering at the Massachusetts Institute of Technology, she worked with an adviser, Nat Durlach, who was studying psychoacoustics—the psychology of hearing—and performing work on hearing-aid technology.

"He was studying people who were both deaf and blind," Tan says. "They have a way of communicating by placing their hands on speakers' faces. They can feel how your mouth opens and closes, the air flow, the muscle tension in the cheeks. They can also pick up laryngeal vibration."

Tan explains that people who are both deaf and blind also place their hands back on their own faces and try to articulate the same sounds until the sensation in their hands felt similar—much the way Helen Keller learned to talk a century earlier.

"We were really amazed and inspired by that method," she says. "That's how I all of a sudden got interested in touch, because I just realized how little we knew about touch, about the richness of touch, and also how much information touch can transmit. Touch can convey speech—that's a lot of information."

Almost Magic

Tan spent 13 years on the Purdue University faculty before joining Microsoft Research in 2011. She is working in two areas of computer haptics: providing key-click feedback on flat keyboards and creating feelings of texture and traction on glass surfaces using electrovibration.

This video is not supported by your browser at this time.
Credit: Microsoft

"The thing that's really, really cool," she says, "is to take a smooth piece of glass but make it feel different—it's almost magic."

The way users currently interact with device screens is entirely two-dimensional, Tan notes, as opposed to a multisensory experience.

Tan's vision, and that of the small but growing field of computer haptics, is to "fully engage the sense of touch in user-device interaction."

Haptics researchers are investigating a range of tactile-feedback mechanisms that involve both hardware and software. Some are based on skin-receptor stimulation, others on muscle-receptor stimulation. The work is heavily interdisciplinary. In the case of vibration feedback, for example, it involves electronic engineering and mechanics, as well as knowledge about human sensitivity to various vibration frequencies.

In the case of haptic keyboards, the key-click feedback is not just symbolic or abstract: It simulates the feeling of pressing a key. One method is to put a layer of material under the glass that bends under electric voltage, which, in turn, bends the glass ever so slightly to simulate a key click.

Tan's Human-Computer Interaction group has developed a number of prototypes that produce haptic keyboard feedback. In a recent research paper, she and Jin Ryong Kim, a graduate student at Purdue, showed that haptic keyboards are far superior to conventional on-screen keyboards in terms of typing speed and accuracy.

Tan's group also is working on haptic approaches that simulate sticky and smooth sensations. This can be accomplished through electrovibration—alternating voltage applied to the glass surface—to change the friction between the fingertip and the screen.

Searching for the Killer App

So how will haptics change the computing experience?

"It's all up to the imagination right now," Tan says. "We haven't really designed anything that has become mainstream enough."

The field has come a long way since 1996, when Tan earned her Ph.D. and had to explain the term "haptics" in her job talks at various computer-science and engineering departments. That was only a couple of years after the first computer-haptics symposium.

"We've made a lot of progress since then, because now everybody has some notion of what haptics is," she says. "At the time, pursuing a career in haptics research was a huge risk."

But in many ways, the field is still in its infancy.

"It's not growing as fast as I would hope," Tan says. "The burden is on those of us who are working on it to really show the value of the work."

One group likely to welcome advances in computer haptics: people with sensory impairments. Tan sees tremendous prospects for haptics to help not only people living with sight loss, but any person.

"My philosophy," she says, "is to design something that's so universal—that's useful to everybody—that everybody benefits."

Tan cites the example of corner cutouts on sidewalk curbs, which were intended to help wheelchair users but have proven valuable to many other people, including delivery drivers with dollies and parents pushing strollers.

"If we really design multimodally, I think everybody benefits," she says. "Then it's truly useful to people with hearing or visual impairments because it's all built into the same product."

Tan has started interacting closely with the Accessibility group at Microsoft, which provides her with insights into how blind people read maps, for example, or navigate a space. She also communicates with designers throughout the company on how haptics might help improve everyday interactions between people and their loved ones over a distance.

"I don't think anybody has the killer app yet," Tan says. "We're just trying lots of different things and seeing how people respond to them."

Explore further: Finding the lost art of Angkor Wat

add to favorites email to friend print save as pdf

Related Stories

Students develop secure new procedure for online banking

Mar 06, 2013

Researchers and students from the University of Tübingen, working with GFT Technologies AG, have developed a new process to make online banking more secure. Their software is being presented this week at the CeBIT computer ...

Tactile technology guaranteed to send shivers down your spine

Aug 08, 2011

A new tactile technology developed at Disney Research, Pittsburgh (DRP), called Surround Haptics, makes it possible for video game players and film viewers to feel a wide variety of sensations, from the smoothness of a finger ...

Finding the lost art of Angkor Wat

Jun 04, 2014

(Phys.org) —Long-lost paintings have been discovered on the walls of Cambodia's ancient Angkor Wat temple, thanks to the keen observations of an ANU researcher.

UltraHaptics—it's magic in the air (w/ Video)

Oct 07, 2013

A system that allows users to experience multi-point haptic feedback above an interactive surface without having to touch or hold any device will be unveiled this week [Friday 11 October] at one of the world's ...

Recommended for you

Skin icons can tap into promise of smartwatch

12 hours ago

You have heard it before: smartwatches are cool wearables but critics remind us of the fact that their small size makes many actions cumbersome and they question how many people will really have them on their ...

Japan firm showcases Bat-Signal of the future

Oct 20, 2014

A free-floating image created by firing lasers into thin air was unveiled in Japan on Monday, offering the possibility one day of projecting messages into a cloudless sky, as seen in Batman.

Do we want an augmented reality or a transformed reality?

Oct 14, 2014

It seems we are headed towards a world where augmented reality (AR) systems will be as common as smartphones are today – it's already about to revolutionise medicine, entertainment, the lives of disabled peop ...

Can it be real? Augmented reality melds work, play

Oct 14, 2014

(AP)—Mark Skwarek is surrounded by infiltrating militants in New York's Central Park. He shoots one, then hearing a noise from behind, spins to take down another. All of a sudden, everything flashes red. ...

User comments : 0