Optical path recognition made audible

Students of Karlsruhe Institute of Technology (KIT) have developed an image recognition algorithm that identifies obstacles and recognizes free pathways. Based on this, the iXpoint software company has now developed the Camassia smartphone app as an assistance system for the visually impaired. It generates acoustic signals that enable users to follow the desired pathway. It is the first interactive assistance system of this type and works independently of satellite navigation, a complex sensor system, or electronic maps.

The need for independent mobility makes the blind and visually impaired face big challenges, also in the 21st century. Diverse technological approaches exist, but most people affected still prefer well-established aids, such as the white stick or a guide dog. One of the reasons is that services available are often based on expensive special hardware. Moreover, many inventions never reach the market, because they are not developed beyond a certain experimental stage. The iXpoint software company, in cooperation with KIT, has a different approach: They have developed an assistance system for the blind that uses the camera and motion sensors of a commercial smartphone. Their app "Camassia" is available for download and enables users to acoustically perceive footpaths.

In 2015, the new navigation method was originally developed for a robot vehicle with which the student group Kamaro Engineering e.V. of KIT took part in an international competition for autonomous robots. "The idea came from an observation that can be verified easily by everyone: Footpaths usually have a lower color saturation than their environment," says informatics student Michael Fürst, who programmed the robot named Beteigeuze at that time. With an image recognition algorithm that converts color information from the on-board camera into control commands, Beteigeuze independently followed the test track. It worked so reliably that Kamaro won the competition with a clear lead. This success encouraged the team to think about other applications. Cooperation with the iXpoint software company then gave rise to the automatic path finder for the visually impaired.

The solution is quite easy to use, says Dr. Sebastian Ritterbusch, project manager of iXpoint, who coordinated the company's development of the app. "The user holds the smartphone in walking direction and as soon as the color properties of the footpath are acquired, you can start walking. You don't need to hold the smartphone straight or calmly." Every second, the algorithm considers 30 individual images that are rectified with the help of the motion sensor installed in the smartphone. On a horizontal axis in front of the user, the algorithm reliably calculates the area with the lowest color saturation and, hence, the most probable direction to follow a footpath. With a maximum delay of one tenth of a second, this information is then represented acoustically. As a standard, the assistance system uses a scale of 24 acoustic halftones. The course of the footpath is indicated by means of stereo sound, sound intensity, and tone height. The type of sonification, however, can be adapted to the specific needs of the user. As an option, white noise or an acoustic representation reduced to the maximum direction probability is available.

The high usability and easy operation of the Camassia assistance system for the visually impaired are also due to computer scientist Gerhard Jaworek of the Study Center for the Visually Impaired (SZS) of KIT. He is blind and accompanied the development from the very beginning: "I helped to ensure that the resulting product is actually helpful." Jaworek points out that the app will never replace his white stick, but is a welcome addition. In particular, he is happy about the interactive applicability at any place. Now, he can also use side paths in the park or find his way in rooms inside.