Optical path recognition made audible

April 20, 2018, Karlsruhe Institute of Technology
Optical path recognition made audible
Image recognition makes the course of a path audible based on variable color saturations. Credit: Harald Kucharek

Students of Karlsruhe Institute of Technology (KIT) have developed an image recognition algorithm that identifies obstacles and recognizes free pathways. Based on this, the iXpoint software company has now developed the Camassia smartphone app as an assistance system for the visually impaired. It generates acoustic signals that enable users to follow the desired pathway. It is the first interactive assistance system of this type and works independently of satellite navigation, a complex sensor system, or electronic maps.

The need for independent mobility makes the blind and visually impaired face big challenges, also in the 21st century. Diverse technological approaches exist, but most people affected still prefer well-established aids, such as the white stick or a guide dog. One of the reasons is that services available are often based on expensive special hardware. Moreover, many inventions never reach the market, because they are not developed beyond a certain experimental stage. The iXpoint software company, in cooperation with KIT, has a different approach: They have developed an assistance system for the blind that uses the camera and motion sensors of a commercial smartphone. Their app "Camassia" is available for download and enables users to acoustically perceive footpaths.

In 2015, the new navigation method was originally developed for a robot vehicle with which the student group Kamaro Engineering e.V. of KIT took part in an international competition for autonomous robots. "The idea came from an observation that can be verified easily by everyone: Footpaths usually have a lower than their environment," says informatics student Michael Fürst, who programmed the robot named Beteigeuze at that time. With an image recognition algorithm that converts color information from the on-board camera into control commands, Beteigeuze independently followed the test track. It worked so reliably that Kamaro won the competition with a clear lead. This success encouraged the team to think about other applications. Cooperation with the iXpoint software company then gave rise to the automatic path finder for the visually impaired.

Optical path recognition made audible
The model Beteigeuze: The robot vehicle uses the same navigation method as the Camassia assistance system; here, at the Field Robot Event 2017. Credit: Christoph Breuner

The solution is quite easy to use, says Dr. Sebastian Ritterbusch, project manager of iXpoint, who coordinated the company's development of the app. "The user holds the smartphone in walking direction and as soon as the color properties of the footpath are acquired, you can start walking. You don't need to hold the smartphone straight or calmly." Every second, the algorithm considers 30 individual images that are rectified with the help of the motion sensor installed in the smartphone. On a horizontal axis in front of the user, the algorithm reliably calculates the area with the lowest color saturation and, hence, the most probable direction to follow a footpath. With a maximum delay of one tenth of a second, this information is then represented acoustically. As a standard, the assistance system uses a scale of 24 acoustic halftones. The course of the footpath is indicated by means of stereo sound, sound intensity, and tone height. The type of sonification, however, can be adapted to the specific needs of the user. As an option, white noise or an acoustic representation reduced to the maximum direction probability is available.

The high usability and easy operation of the Camassia assistance system for the visually impaired are also due to computer scientist Gerhard Jaworek of the Study Center for the Visually Impaired (SZS) of KIT. He is blind and accompanied the development from the very beginning: "I helped to ensure that the resulting product is actually helpful." Jaworek points out that the app will never replace his white stick, but is a welcome addition. In particular, he is happy about the interactive applicability at any place. Now, he can also use side paths in the park or find his way in rooms inside.

Credit: Karlsruhe Institute of Technology

Explore further: Safe navigation for visually impaired persons

Related Stories

Safe navigation for visually impaired persons

September 9, 2013

Blind and visually impaired persons find it very difficult to move about in unknown environments. The white canes used by them are still the preferred tools for orientation. The project "A Mobility and Navigational Aid for ...

Developing a tool to increase mobility for the blind

October 26, 2017

People with vision impairments face a perpetual problem: maneuvering through a world filled with obstacles and hazards. Meet Good Vibrations, a team of CU Boulder electrical engineering students who created Pulse, a product ...

App improves the safety of blind pedestrians in cities

January 22, 2015

Siemens is developing a system that helps blind and visually impaired people walk safely through cities. In cooperation with the Technical University of Braunschweig and several partners, Siemens is working on a comprehensive ...

Wearable system helps visually impaired users navigate

May 31, 2017

Computer scientists have been working for decades on automatic navigation systems to aid the visually impaired, but it's been difficult to come up with anything as reliable and easy to use as the white cane, the type of metal-tipped ...

Recommended for you


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.