iPhone app uses crowd sourcing to help the blind

May 13, 2011 by Bob Yirka report

(PhysOrg.com) -- A team of innovative thinkers from several universities working together on a joint project to merge existing technology (the iPhone) with a real-world crowd sourcing application to help blind people make decisions about their environment, has led to VizWiz, an iPhone/Mechanical Turk application that allows a blind user to snap a photo of something, pose a question about something shown in the photograph, and then receive an answer back within seconds.

Winner of the best paper prize last year at ACM’s 23rd symposium on User Interface Software and Technology, VizWiz sidesteps the problem of having a computer try to decipher such problems as reading can labels for blind people, by handing the chore off to a crowd of awaiting workers who look at a the user has taken, listen to a question about it from them, then respond by recording an answer, which is then relayed back to the original user.

Mechanical Turk is an Amazon web site that pays workers (or turkers as they’re called) to perform oftentimes routine or mundane tasks that are easy for humans to do, but difficult for computers. Tasks are generally short and very low paying, but turkers make up for that by doing a lot of them.

The downside to Mechanical Turk though, is that it’s not very user friendly for those that want the answers from the system; this is where scientists from MIT came in, they developed a toolkit called Turkit that works as a go-between, allowing other programmers to create applications that utilize Mechanical Turk, without having to actually visit the site. This is what the developers of VizWiz (led by the University of Rochester’s, Jeffrey Bigham) have done. Instead of forcing a blind client to have to visit and use Mechanical Turk directly, they’ve created an that works in conjunction with a Turkit application, which in turn sends requests to Mechanical Turk.

This video is not supported by your browser at this time.

But, that’s not all; to solve the problem of users having to wait a long time for an answer, the Turkit application contacts the Mechanical Turk workers as soon as the app is launched, readying them for the work that will soon be heading their way, which means they will be in a position to answer the question sent, as soon as the picture and audio recording arrive.
Unfortunately, the VizWiz is still under development, so it’s not yet available for general use, but that day will arrive eventually, and when it does, it’s likely to prove a boon to blind and vision impaired people who will likely find it, as one of the test volunteers described, “…very useful, because I get so frustrated when I need sighted help and no one is there."

Explore further: Tecnalia designs an app to help elderly people get around on public transport

More information: VizWiz: hci.cs.rochester.edu/currentprojects.php?proj=vw

Related Stories

Programming the crowds

Oct 27, 2010

At the Association for Computing Machinery’s 23rd symposium on User Interface Software and Technology in October, members of  the User Interface Design Group at MIT’s Computer Science and Artificial ...

Better hearing just an App away

May 02, 2011

(Medical Xpress) -- People with hearing problems living in the developing world may have a solution on the horizon, thanks to an innovative University of Alberta professor in the Faculty of Rehabilitation Medicine.

Scientist creates 3-D scanner iPhone app (w/ video)

Apr 14, 2011

(PhysOrg.com) -- Leave it to an iPhone app developer to turn a tool that cost hundreds of dollars a year ago into something that can be done with a 99-cent app. Grant Schindler, research scientist in Georgia ...

iPhone and iPad with iOS 4 records your moves (w/video)

Apr 20, 2011

(PhysOrg.com) -- Security researchers have discovered that any iPhone or iPad that has been updated with iOS 4 records everywhere you have been to a secret file. The file is also copied to the owner’s ...

XWave for iPhone lets you read your own mind

Jan 12, 2011

(PhysOrg.com) -- A new application for the iPhone, the XWave, lets you read your own mind via a headset clamped to your head and connected to the phone’s audio jack.

Recommended for you

Google worker shows early-draft glimpse of Chrome OS

Jul 20, 2014

The Chrome OS is in for a future look. Athena, a Chromium OS project, will bring forth the new Chrome OS user experience. Google's François Beaufort on Friday, referring to the screenshot he posted, said," ...

Google eyes Chrome on Windows laptop battery drain

Jul 19, 2014

Google Chrome on Microsoft Windows has been said to have a problem for some time but this week comes news that Google will give it the attention others think the problem quite deserves. Namely, Google is to ...

Mental-health monitoring goes mobile

Jul 16, 2014

Behavioral health analytics startup Ginger.io sees smartphones as "automated diaries" containing valuable insight into the mental well-being of people with mental illnesses.

User comments : 0