Smartphone app helps people with blindness navigate their surroundings

Previously, when Chieko Asakawa navigated her way across the Carnegie Mellon University campus, she used her white cane to identify obstructions with her ears alert to recognizable sounds and intuition in full power to keep track of her location.

She is blind. So the IBM fellow from Japan and visiting faculty member at the university's Robotics Institute would count steps and use known landmarks to document her progress. But if someone stopped to chat or bumped into her, she lost her step count and became disoriented.

"You have to always concentrate," she said. "For us, you are always thinking how many steps to the next turn and where you are. You can't walk while thinking about other things. It's very easy to get lost if you are not careful."

But this week, Asakawa walked with her long, white cane through Newell-Simon Hall, including an elevator ride, to an ATM machine in Ween Hall, with assistance from the NavCog app downloaded onto her smartphone. Like a GPS system it provided a voice that described distances, what directions she was to turn and when she arrived at landmarks and the ATM machine.

NavCog, which she developed with help from Kris Kitani at Carnegie Mellon's Robotics Institute and IBM, now is available for free at the App Store downloads on iTunes. It helps people with to find their way indoors and out. While still in pilot phase, the app and its open-source technology allows others to advance the technology and expand areas where it can be used.

NavCog uses Bluetooth sensors placed at turns, intersections and important destinations (elevators, entrances or ATM machines) to provide one's position within 3 to 5 feet of actual location. Using a map editing tool and localization algorithms, it can identify your location almost in , which direction you are facing and additional information about your surroundings.

It allows Asakawa, who holds a Ph.D. in engineering, to walk unassisted through various buildings and sections of the campus based on signals from 200 battery-powered sensors. There's still the occasional bump, brush-by or problem determining whether the elevator is going up or down, but the campus system shows its potential.

"While visually impaired people like myself have become independent online, we are still challenged in the real world," she said, adding that the app can accelerate research and serve as the basis for more useful and expanded navigational system for people with blindness.

Already she and Kitani, a doctor of robotics, are testing a small camera hung over her ear and using emotion-recognition software to describe the emotional state of people approaching her. In time, computer-vision technology will compare previously downloaded images of an area with ones generated in real time by the smartphone to identify one's location more accurately without sensors.

Elaine R. Welch, president and CEO of the Pennsylvania Association for the Blind, said people with visual impairment already use GPS technology to provide their general location outside. GPS cannot be used indoors. Such technology has potential to improve the quality of life of people with blindness, with self-driving cars representing the Holy Grail.

"I've been around since 1982, and I see what technology has done for people to achieve independence. It's just been phenomenal and this is great for people with disabilities," she said. "I really applaud (the Carnegie Mellon-IBM team) for doing this. Anything that can help people to be independent and more sure of themselves, whether they are blind or sighted in unfamiliar places, is great."

Explore further

Open platform to help the blind navigate surroundings

©2015 Pittsburgh Post-Gazette
Distributed by Tribune Content Agency, LLC.

Citation: Smartphone app helps people with blindness navigate their surroundings (2015, October 27) retrieved 6 December 2021 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Feedback to editors