App turns tablet into math aid for visually impaired students (w/ video)

March 5, 2012 by David Salisbury
A visually impaired student traces the lines on a grid using her sense of touch. Credit: Pat Slattery / Vanderbilt University

Without looking down, Kira runs her index finger across the screen of an Android tablet that she is holding in her lap. For the occasion, she has painted her fingernails bright pink. When her finger touches a line drawn on the screen, the tablet vibrates quietly. Scanning her finger back and forth and feeling the vibration come and go allows her to trace the line's path. When her finger reaches a pink dot, the tablet gives off an electronic tone and she grins delightedly.

Kira is one of two visually impaired who are testing a new Android app, one designed to assist students like her in mastering algebra, , graphing and other subjects that are particularly hard to comprehend without the aid of normal vision.

The is the brainstorm of Jenna Gorlewicz, a graduate student in the Medical and Electromechanical Design Laboratory (MED Lab) at Vanderbilt University, and her advisor Robert Webster, an assistant professor of , who directs the lab. Given the enthusiastic reaction of Kira, her Quinn and their teacher, her innovation could have a major impact on how science, technology, engineering and math – the critical STEM subjects – are taught to the visually impaired.

The video will load shortly

A major change in direction

The project is a considerable change from the research projects that Gorlewicz pursued during her first two years in the MED Lab. Supported by a fellowship from the National Science Foundation, she worked on improving cochlear implant surgery and miniaturizing endoscopic robotic capsules. But she wasn't completely satisfied: "I'm a real people person," she explained. "So I wanted a project that would let me interact directly with the people it was helping."

"Jenna kept saying that she wanted to work more with people, so we began exploring other possible projects," said Webster. "Fortunately, her fellowship and my NSF CAREER Award gave us enough flexibility to think outside the box and explore a wide variety of thesis topics for her."

One of the ideas they came up with was a study of the effectiveness of haptic technology: technology that takes advantage of a user's sense of touch by applying forces, vibrations or motions to enhance remote control of machines, devices or virtual objects. The lab was experimenting with haptic interfaces to control some of their biomedical robots.

Matching haptics with visually impaired

"When I began reading articles about haptic technology being incorporated into these new touch screen devices, I realized that the people who really need haptics are people with impaired vision because they heavily rely on their sense of touch to 'see' the world around them." Gorlewicz said. "I love math and I love teaching, so I immediately thought of using them for math education, because it has such a strong visual component." After she did some research into the methods that are currently used to teach mathematics to the visually impaired, she became even more enthusiastic about her idea.

When she started the new project 18 months ago, the only off-the-shelf tablet that Gorlewicz could find that included tactile feedback was a $2,500 model produced for industrial and commercial applications. Since then haptic feedback has been added in a number of consumer tablets that sell for as little as $300.

The grad student has programmed these tablets so they vibrate or generate a specific tone when the student's fingertip touches a line, curve or shape displayed on the screen. The devices can generate vibrations with a number of different frequencies and hundreds of different sounds. This allows Gorlewicz to assign different tactile or audio signals to different features. For example, in an exercise that includes an X-Y grid, she can set the horizontal and vertical lines to vibrate at different frequencies and set points to make a certain tone. In this way, it's easier for the students to distinguish between the gridlines and the points on the grid.

"If one of these tablets were networked wirelessly to the teacher's computer, then, when he or she projects a graph or equation on the screen at the front of the class, the same graph would appear on the student's tablet. They could then use their senses of touch and hearing to follow the content the teacher is discussing," Gorlewicz said.

"When Jenna first approached me with the idea, I thought it would be interesting and might be some small help," said Ann Smith, the teacher of the visually impaired who works with Kira and Quinn at Hillsboro High School in Nashville. "The more experience I have with it, the more valuable I think it could be. It makes the work more accessible. The students are really interested and they talk about it even when Jenna isn't here."

Hillsboro High follows the approach of including students with special needs in classes with their sighted peers as much as possible. That means Kira and Quinn attend general education classes. Smith sits with them during their math classes and uses various methods to ensure that they can follow the teacher's presentation. With Quinn, who can read Braille and see large print, this involves duplicating the shapes and figures the teacher is using on a small white board that he can hold close enough to see. In Kira's case Smith must use tactile aids like pins and pipe-cleaners on a bulletin board with raised graphing lines to reproduce complex mathematical shapes that Kira can feel with her fingers.

"Because this takes a certain amount of time, we are always one or two steps behind the teacher," Smith said. "One of these haptic tablets would allow them to keep up much better. If I didn't have to attend class with them, it would also make them feel more independent."

Smith also pointed out that such a tablet can be used by students with widely differing degrees of impairment because it combines visual, tactile and audio capabilities.

Some learning required

According to Kira and Quinn, there is a little bit of a learning curve with the new device. "At first, I didn't think this would help me," Quinn said. "But after I started using it I found that it can be very helpful." According to Kira, "The biggest obstacle was getting the correct mental images. However, once you get the knack, it gets fairly easy."

"It was amazing how quickly they caught on and how good they have become," Gorlewicz said.

According to the graduate student, every time she meets with the students, they come up with new features that they would like her to add. The configuration that they think would be the most useful is a touch-sensitive version of the graphing calculators that are widely used by their sighted classmates.

"It would really help to have something like this because it makes us equal to everyone else," Kira said.

Explore further: New device puts vision impaired in the picture

Related Stories

New device puts vision impaired in the picture

April 28, 2011

( -- Visually impaired people may soon have greater access to graphical information thanks to a new device developed by Monash University’s Faculty of Information and Technology.

A touchscreen you can really feel (w/ video)

November 16, 2011

( -- Swiss researchers have invented a new generation of tactile surfaces with relief effects – users can feel actual raised keys under their fingers. This technology could have many applications, particularly ...

'Sighted' wheelchair tested

May 17, 2011

Research on an electric wheelchair that can sense it's environment and transmit information to a person who is visually impaired, has been tested at Lulea University of Technology, Sweden. Daniel Innala Ahlmark, a prospective ...

Enabling the blind to find their way

October 24, 2008

( -- “Eyes on the future” is the mantra of the ‘World Sight Day’ held this month to raise awareness of blindness and vision impairment. New technologies, developed by European researchers offering the ...

Georgia Tech develops braille-like texting app (w/ video)

February 17, 2012

Imagine if smartphone and tablet users could text a note under the table during a meeting without anyone being the wiser. Mobile gadget users might also be enabled to text while walking, watching TV or socializing without ...

Recommended for you

Light beam replaces blood test during heart surgery

February 27, 2017

A University of Central Florida professor has invented a way to use light to continuously monitor a surgical patient's blood, for the first time providing a real-time status during life-and-death operations.

5 bn mobile phone users in 2017: study

February 27, 2017

The number of mobile phone users globally will surpass five billion by the middle of this year, according to a study released Monday by GSMA, the association of mobile operators.

Tracking the movement of cyborg cockroaches

February 27, 2017

New research from North Carolina State University offers insights into how far and how fast cyborg cockroaches - or biobots - move when exploring new spaces. The work moves researchers closer to their goal of using biobots ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.