iPhone app uses crowd sourcing to help the blind

(PhysOrg.com) -- A team of innovative thinkers from several universities working together on a joint project to merge existing technology (the iPhone) with a real-world crowd sourcing application to help blind people make decisions about their environment, has led to VizWiz, an iPhone/Mechanical Turk application that allows a blind user to snap a photo of something, pose a question about something shown in the photograph, and then receive an answer back within seconds.

Winner of the best paper prize last year at ACM’s 23rd symposium on User Interface Software and Technology, VizWiz sidesteps the problem of having a computer try to decipher such problems as reading can labels for blind people, by handing the chore off to a crowd of awaiting workers who look at a photograph the user has taken, listen to a question about it from them, then respond by recording an answer, which is then relayed back to the original user.

Mechanical Turk is an Amazon web site that pays workers (or turkers as they’re called) to perform oftentimes routine or mundane tasks that are easy for humans to do, but difficult for computers. Tasks are generally short and very low paying, but turkers make up for that by doing a lot of them.

The downside to Mechanical Turk though, is that it’s not very user friendly for those that want the answers from the system; this is where scientists from MIT came in, they developed a toolkit called Turkit that works as a go-between, allowing other programmers to create applications that utilize Mechanical Turk, without having to actually visit the site. This is what the developers of VizWiz (led by the University of Rochester’s, Jeffrey Bigham) have done. Instead of forcing a blind client to have to visit and use Mechanical Turk directly, they’ve created an iPhone app that works in conjunction with a Turkit application, which in turn sends requests to Mechanical Turk.

But, that’s not all; to solve the problem of users having to wait a long time for an answer, the Turkit application contacts the Mechanical Turk workers as soon as the iPhone app is launched, readying them for the work that will soon be heading their way, which means they will be in a position to answer the question sent, as soon as the picture and audio recording arrive.
Unfortunately, the VizWiz is still under development, so it’s not yet available for general use, but that day will arrive eventually, and when it does, it’s likely to prove a boon to blind and vision impaired people who will likely find it, as one of the test volunteers described, “…very useful, because I get so frustrated when I need sighted help and no one is there."