iPhone app uses crowd sourcing to help the blind

May 13, 2011 by Bob Yirka report

(PhysOrg.com) -- A team of innovative thinkers from several universities working together on a joint project to merge existing technology (the iPhone) with a real-world crowd sourcing application to help blind people make decisions about their environment, has led to VizWiz, an iPhone/Mechanical Turk application that allows a blind user to snap a photo of something, pose a question about something shown in the photograph, and then receive an answer back within seconds.

Winner of the best paper prize last year at ACM’s 23rd symposium on User Interface Software and Technology, VizWiz sidesteps the problem of having a computer try to decipher such problems as reading can labels for blind people, by handing the chore off to a crowd of awaiting workers who look at a the user has taken, listen to a question about it from them, then respond by recording an answer, which is then relayed back to the original user.

Mechanical Turk is an Amazon web site that pays workers (or turkers as they’re called) to perform oftentimes routine or mundane tasks that are easy for humans to do, but difficult for computers. Tasks are generally short and very low paying, but turkers make up for that by doing a lot of them.

The downside to Mechanical Turk though, is that it’s not very user friendly for those that want the answers from the system; this is where scientists from MIT came in, they developed a toolkit called Turkit that works as a go-between, allowing other programmers to create applications that utilize Mechanical Turk, without having to actually visit the site. This is what the developers of VizWiz (led by the University of Rochester’s, Jeffrey Bigham) have done. Instead of forcing a blind client to have to visit and use Mechanical Turk directly, they’ve created an that works in conjunction with a Turkit application, which in turn sends requests to Mechanical Turk.

This video is not supported by your browser at this time.

But, that’s not all; to solve the problem of users having to wait a long time for an answer, the Turkit application contacts the Mechanical Turk workers as soon as the app is launched, readying them for the work that will soon be heading their way, which means they will be in a position to answer the question sent, as soon as the picture and audio recording arrive.
Unfortunately, the VizWiz is still under development, so it’s not yet available for general use, but that day will arrive eventually, and when it does, it’s likely to prove a boon to blind and vision impaired people who will likely find it, as one of the test volunteers described, “…very useful, because I get so frustrated when I need sighted help and no one is there."

Explore further: Android gains in US, basic phones almost extinct

More information: VizWiz: hci.cs.rochester.edu/currentprojects.php?proj=vw

Related Stories

Programming the crowds

Oct 27, 2010

At the Association for Computing Machinery’s 23rd symposium on User Interface Software and Technology in October, members of  the User Interface Design Group at MIT’s Computer Science and Artificial ...

Better hearing just an App away

May 02, 2011

(Medical Xpress) -- People with hearing problems living in the developing world may have a solution on the horizon, thanks to an innovative University of Alberta professor in the Faculty of Rehabilitation Medicine.

Scientist creates 3-D scanner iPhone app (w/ video)

Apr 14, 2011

(PhysOrg.com) -- Leave it to an iPhone app developer to turn a tool that cost hundreds of dollars a year ago into something that can be done with a 99-cent app. Grant Schindler, research scientist in Georgia ...

iPhone and iPad with iOS 4 records your moves (w/video)

Apr 20, 2011

(PhysOrg.com) -- Security researchers have discovered that any iPhone or iPad that has been updated with iOS 4 records everywhere you have been to a secret file. The file is also copied to the owner’s ...

XWave for iPhone lets you read your own mind

Jan 12, 2011

(PhysOrg.com) -- A new application for the iPhone, the XWave, lets you read your own mind via a headset clamped to your head and connected to the phone’s audio jack.

Recommended for you

Android gains in US, basic phones almost extinct

Apr 18, 2014

The Google Android platform grabbed the majority of mobile phones in the US market in early 2014, as consumers all but abandoned non-smartphone handsets, a survey showed Friday.

Hackathon team's GoogolPlex gives Siri extra powers

Apr 17, 2014

(Phys.org) —Four freshmen at the University of Pennsylvania have taken Apple's personal assistant Siri to behave as a graduate-level executive assistant which, when asked, is capable of adjusting the temperature ...

Microsoft CEO is driving data-culture mindset

Apr 16, 2014

(Phys.org) —Microsoft's future strategy: is all about leveraging data, from different sources, coming together using one cohesive Microsoft architecture. Microsoft CEO Satya Nadella on Tuesday, both in ...

User comments : 0

More news stories

Ex-Apple chief plans mobile phone for India

Former Apple chief executive John Sculley, whose marketing skills helped bring the personal computer to desktops worldwide, says he plans to launch a mobile phone in India to exploit its still largely untapped ...

Airbnb rental site raises $450 mn

Online lodging listings website Airbnb inked a $450 million funding deal with investors led by TPG, a source close to the matter said Friday.

Health care site flagged in Heartbleed review

People with accounts on the enrollment website for President Barack Obama's signature health care law are being told to change their passwords following an administration-wide review of the government's vulnerability to the ...

A homemade solar lamp for developing countries

(Phys.org) —The solar lamp developed by the start-up LEDsafari is a more effective, safer, and less expensive form of illumination than the traditional oil lamp currently used by more than one billion people ...

NASA's space station Robonaut finally getting legs

Robonaut, the first out-of-this-world humanoid, is finally getting its space legs. For three years, Robonaut has had to manage from the waist up. This new pair of legs means the experimental robot—now stuck ...

Filipino tests negative for Middle East virus

A Filipino nurse who tested positive for the Middle East virus has been found free of infection in a subsequent examination after he returned home, Philippine health officials said Saturday.

Egypt archaeologists find ancient writer's tomb

Egypt's minister of antiquities says a team of Spanish archaeologists has discovered two tombs in the southern part of the country, one of them belonging to a writer and containing a trove of artifacts including reed pens ...