iPhone app uses crowd sourcing to help the blind

May 13, 2011 by Bob Yirka report

(PhysOrg.com) -- A team of innovative thinkers from several universities working together on a joint project to merge existing technology (the iPhone) with a real-world crowd sourcing application to help blind people make decisions about their environment, has led to VizWiz, an iPhone/Mechanical Turk application that allows a blind user to snap a photo of something, pose a question about something shown in the photograph, and then receive an answer back within seconds.

Winner of the best paper prize last year at ACM’s 23rd symposium on User Interface Software and Technology, VizWiz sidesteps the problem of having a computer try to decipher such problems as reading can labels for blind people, by handing the chore off to a crowd of awaiting workers who look at a the user has taken, listen to a question about it from them, then respond by recording an answer, which is then relayed back to the original user.

Mechanical Turk is an Amazon web site that pays workers (or turkers as they’re called) to perform oftentimes routine or mundane tasks that are easy for humans to do, but difficult for computers. Tasks are generally short and very low paying, but turkers make up for that by doing a lot of them.

The downside to Mechanical Turk though, is that it’s not very user friendly for those that want the answers from the system; this is where scientists from MIT came in, they developed a toolkit called Turkit that works as a go-between, allowing other programmers to create applications that utilize Mechanical Turk, without having to actually visit the site. This is what the developers of VizWiz (led by the University of Rochester’s, Jeffrey Bigham) have done. Instead of forcing a blind client to have to visit and use Mechanical Turk directly, they’ve created an that works in conjunction with a Turkit application, which in turn sends requests to Mechanical Turk.

This video is not supported by your browser at this time.

But, that’s not all; to solve the problem of users having to wait a long time for an answer, the Turkit application contacts the Mechanical Turk workers as soon as the app is launched, readying them for the work that will soon be heading their way, which means they will be in a position to answer the question sent, as soon as the picture and audio recording arrive.
Unfortunately, the VizWiz is still under development, so it’s not yet available for general use, but that day will arrive eventually, and when it does, it’s likely to prove a boon to blind and vision impaired people who will likely find it, as one of the test volunteers described, “…very useful, because I get so frustrated when I need sighted help and no one is there."

Explore further: Apple gives beta users a peek at OS X Yosemite

More information: VizWiz: hci.cs.rochester.edu/currentprojects.php?proj=vw

Related Stories

Programming the crowds

Oct 27, 2010

At the Association for Computing Machinery’s 23rd symposium on User Interface Software and Technology in October, members of  the User Interface Design Group at MIT’s Computer Science and Artificial ...

Better hearing just an App away

May 02, 2011

(Medical Xpress) -- People with hearing problems living in the developing world may have a solution on the horizon, thanks to an innovative University of Alberta professor in the Faculty of Rehabilitation Medicine.

Scientist creates 3-D scanner iPhone app (w/ video)

Apr 14, 2011

(PhysOrg.com) -- Leave it to an iPhone app developer to turn a tool that cost hundreds of dollars a year ago into something that can be done with a 99-cent app. Grant Schindler, research scientist in Georgia ...

iPhone and iPad with iOS 4 records your moves (w/video)

Apr 20, 2011

(PhysOrg.com) -- Security researchers have discovered that any iPhone or iPad that has been updated with iOS 4 records everywhere you have been to a secret file. The file is also copied to the owner’s ...

XWave for iPhone lets you read your own mind

Jan 12, 2011

(PhysOrg.com) -- A new application for the iPhone, the XWave, lets you read your own mind via a headset clamped to your head and connected to the phone’s audio jack.

Recommended for you

User comments : 0