App turns tablet into math aid for visually impaired students (w/ video)

Mar 05, 2012 by David Salisbury
A visually impaired student traces the lines on a grid using her sense of touch. Credit: Pat Slattery / Vanderbilt University

Without looking down, Kira runs her index finger across the screen of an Android tablet that she is holding in her lap. For the occasion, she has painted her fingernails bright pink. When her finger touches a line drawn on the screen, the tablet vibrates quietly. Scanning her finger back and forth and feeling the vibration come and go allows her to trace the line's path. When her finger reaches a pink dot, the tablet gives off an electronic tone and she grins delightedly.

Kira is one of two visually impaired who are testing a new Android app, one designed to assist students like her in mastering algebra, , graphing and other subjects that are particularly hard to comprehend without the aid of normal vision.

The is the brainstorm of Jenna Gorlewicz, a graduate student in the Medical and Electromechanical Design Laboratory (MED Lab) at Vanderbilt University, and her advisor Robert Webster, an assistant professor of , who directs the lab. Given the enthusiastic reaction of Kira, her Quinn and their teacher, her innovation could have a major impact on how science, technology, engineering and math – the critical STEM subjects – are taught to the visually impaired.

This video is not supported by your browser at this time.

A major change in direction

The project is a considerable change from the research projects that Gorlewicz pursued during her first two years in the MED Lab. Supported by a fellowship from the National Science Foundation, she worked on improving cochlear implant surgery and miniaturizing endoscopic robotic capsules. But she wasn't completely satisfied: "I'm a real people person," she explained. "So I wanted a project that would let me interact directly with the people it was helping."

"Jenna kept saying that she wanted to work more with people, so we began exploring other possible projects," said Webster. "Fortunately, her fellowship and my NSF CAREER Award gave us enough flexibility to think outside the box and explore a wide variety of thesis topics for her."

One of the ideas they came up with was a study of the effectiveness of haptic technology: technology that takes advantage of a user's sense of touch by applying forces, vibrations or motions to enhance remote control of machines, devices or virtual objects. The lab was experimenting with haptic interfaces to control some of their biomedical robots.

Matching haptics with visually impaired

"When I began reading articles about haptic technology being incorporated into these new touch screen devices, I realized that the people who really need haptics are people with impaired vision because they heavily rely on their sense of touch to 'see' the world around them." Gorlewicz said. "I love math and I love teaching, so I immediately thought of using them for math education, because it has such a strong visual component." After she did some research into the methods that are currently used to teach mathematics to the visually impaired, she became even more enthusiastic about her idea.

When she started the new project 18 months ago, the only off-the-shelf tablet that Gorlewicz could find that included tactile feedback was a $2,500 model produced for industrial and commercial applications. Since then haptic feedback has been added in a number of consumer tablets that sell for as little as $300.

The grad student has programmed these tablets so they vibrate or generate a specific tone when the student's fingertip touches a line, curve or shape displayed on the screen. The devices can generate vibrations with a number of different frequencies and hundreds of different sounds. This allows Gorlewicz to assign different tactile or audio signals to different features. For example, in an exercise that includes an X-Y grid, she can set the horizontal and vertical lines to vibrate at different frequencies and set points to make a certain tone. In this way, it's easier for the students to distinguish between the gridlines and the points on the grid.

"If one of these tablets were networked wirelessly to the teacher's computer, then, when he or she projects a graph or equation on the screen at the front of the class, the same graph would appear on the student's tablet. They could then use their senses of touch and hearing to follow the content the teacher is discussing," Gorlewicz said.

"When Jenna first approached me with the idea, I thought it would be interesting and might be some small help," said Ann Smith, the teacher of the visually impaired who works with Kira and Quinn at Hillsboro High School in Nashville. "The more experience I have with it, the more valuable I think it could be. It makes the work more accessible. The students are really interested and they talk about it even when Jenna isn't here."

Hillsboro High follows the approach of including students with special needs in classes with their sighted peers as much as possible. That means Kira and Quinn attend general education classes. Smith sits with them during their math classes and uses various methods to ensure that they can follow the teacher's presentation. With Quinn, who can read Braille and see large print, this involves duplicating the shapes and figures the teacher is using on a small white board that he can hold close enough to see. In Kira's case Smith must use tactile aids like pins and pipe-cleaners on a bulletin board with raised graphing lines to reproduce complex mathematical shapes that Kira can feel with her fingers.

"Because this takes a certain amount of time, we are always one or two steps behind the teacher," Smith said. "One of these haptic tablets would allow them to keep up much better. If I didn't have to attend class with them, it would also make them feel more independent."

Smith also pointed out that such a tablet can be used by students with widely differing degrees of impairment because it combines visual, tactile and audio capabilities.

Some learning required

According to Kira and Quinn, there is a little bit of a learning curve with the new device. "At first, I didn't think this would help me," Quinn said. "But after I started using it I found that it can be very helpful." According to Kira, "The biggest obstacle was getting the correct mental images. However, once you get the knack, it gets fairly easy."

"It was amazing how quickly they caught on and how good they have become," Gorlewicz said.

According to the graduate student, every time she meets with the students, they come up with new features that they would like her to add. The configuration that they think would be the most useful is a touch-sensitive version of the graphing calculators that are widely used by their sighted classmates.

"It would really help to have something like this because it makes us equal to everyone else," Kira said.

Explore further: Researchers propose network-based evaluation tool to assess relief operations feasibility

Related Stories

New device puts vision impaired in the picture

Apr 28, 2011

(PhysOrg.com) -- Visually impaired people may soon have greater access to graphical information thanks to a new device developed by Monash University’s Faculty of Information and Technology.

A touchscreen you can really feel (w/ video)

Nov 16, 2011

(PhysOrg.com) -- Swiss researchers have invented a new generation of tactile surfaces with relief effects – users can feel actual raised keys under their fingers. This technology could have many applications, particularly ...

'Sighted' wheelchair tested

May 17, 2011

Research on an electric wheelchair that can sense it's environment and transmit information to a person who is visually impaired, has been tested at Lulea University of Technology, Sweden. Daniel Innala Ahlmark, ...

Enabling the blind to find their way

Oct 24, 2008

(PhysOrg.com) -- “Eyes on the future” is the mantra of the ‘World Sight Day’ held this month to raise awareness of blindness and vision impairment. New technologies, developed by European researchers offering the ...

Georgia Tech develops braille-like texting app (w/ video)

Feb 17, 2012

Imagine if smartphone and tablet users could text a note under the table during a meeting without anyone being the wiser. Mobile gadget users might also be enabled to text while walking, watching TV or socializing ...

Recommended for you

Large streams of data warn cars, banks and oil drillers

4 hours ago

Better warning systems that alert motorists to a collision, make banks aware of the risk of losses on bad customers, and tell oil companies about potential problems with new drilling. This is the aim of AMIDST, the EU project ...

User comments : 0

More news stories

Quantenna promises 10-gigabit Wi-Fi by next year

(Phys.org) —Quantenna Communications has announced that it has plans for releasing a chipset that will be capable of delivering 10Gbps WiFi to/from routers, bridges and computers by sometime next year. ...

Unlocking secrets of new solar material

(Phys.org) —A new solar material that has the same crystal structure as a mineral first found in the Ural Mountains in 1839 is shooting up the efficiency charts faster than almost anything researchers have ...

Floating nuclear plants could ride out tsunamis

When an earthquake and tsunami struck the Fukushima Daiichi nuclear plant complex in 2011, neither the quake nor the inundation caused the ensuing contamination. Rather, it was the aftereffects—specifically, ...

New US-Spanish firm says targets rich mobile ad market

Spanish telecoms firm Telefonica and US investment giant Blackstone launched a mobile telephone advertising venture on Wednesday, challenging internet giants such as Google and Facebook in a multi-billion-dollar ...