Research could ensure that crowd work becomes a career option, not a dead end

February 7, 2013, Carnegie Mellon University

Crowdsourcing is an effective way to mobilize people to accomplish tasks on a global scale, but some researchers fear that crowd work for pay could easily become the high-tech equivalent of a sweat shop. Trivial work for rock bottom pay isn't inevitable, however, and they've outlined a research agenda to make crowd work both intellectually and monetarily rewarding.

Leading researchers in crowd work from Carnegie Mellon University and other institutions will present their plan, hashed out in a special workshop last spring, at the Association for Computing Machinery's Conference on Computer Supported Cooperative Work and , CSCW 2013, Feb. 27 in San Antonio, Texas.

Finding ways to enhance collaboration, incorporate artificial intelligence and create ways for workers to build reputations are among the research challenges ahead.

"When my baby daughter was born I asked myself, 'would I be proud to see her grow up to be a crowd worker?'" said Aniket Kittur, assistant professor in Carnegie Mellon's Human-Computer Interaction Institute.

Co-authors with Kittur of the research strategy include Jeffrey Nickerson, director of the Center for Decision Technologies at Stevens Institute of Technology, and Michael Bernstein, assistant professor of computer science at Stanford University. Other leading crowd work researchers from Carnegie Mellon, Northwestern University and the University of Texas, Austin also contributed to the report, which is available for download.

The crowd work industry has expanded rapidly in recent years, with a number of vendors now offering work for people who get paid per task or who compete for prizes. A prominent vendor is Amazon Mechanical Turk, which claims more than 500,000 workers in more than 190 countries who complete tasks that, in some cases, may take only seconds to perform. Another, CrowdFlower, says it can access more than 2 million contributors worldwide. Others, such as oDesk, provide skilled labor, including , designers and translators. Platforms such as Innocentive invite people to invent solutions to problems in hopes of winning a prize.

The still-young industry could grow very large because portions of almost any job—perhaps as much as 20 percent—potentially can be sent "down the wire," Kittur said. Many crowd workers are paid substantially less than U.S. minimum wage, however, and, left to market forces, the crowd workforce could remain stigmatized and exploited.

"What if I want access to the best people in the world, but for only five minutes of their time?" Kittur said. As beneficial as that might be for some businesses, that possibility will not be achieved if the crowd workplace isn't attractive for the very best workers and thinkers, he added. The call for action by Kittur and his colleagues, also discussed in a recent post on the Follow the Crowd blog, envisions three major research steps:

  • Create career ladders. Research is needed in how to structure teams so that skilled workers can train novices, as well as help design jobs and catch problems. Mechanisms are needed for credentialing workers. A better understanding of worker motivations could lead to better job designs.
  • Improve task design through better communication. Research suggests that some quality problems in crowd work have more to do with poorly designed tasks than with unskilled workers. could be used in complex tasks to identify work products that might still need improvement and assign workers accordingly. The crowd itself also may be used to train the computer programs, helping them support a broader range of tasks. Improved instructions and feedback mechanisms likewise could improve the work product.
  • Enable learning. Quality assurance assessments can identify skills that workers need to polish or learn to tackle new work tasks. Online tutoring, combined with tracking of work history, could support personalized instruction and feedback. The work platforms themselves will need mechanisms for learning what kinds of work requests attract talented workers, recognizing the patterns of learning and skill building among workers and determining what tasks are appropriate for which types of .

Explore further: Workforce from the digital cloud

Related Stories

Workforce from the digital cloud

February 29, 2012

By means of cloud computing, enterprises can access scalable computing power and storage capacity. A people cloud, by contrast, supplies a scalable number of workers via the internet. It is used when non-automated tasks are ...

iPhone app uses crowd sourcing to help the blind

May 13, 2011

( -- A team of innovative thinkers from several universities working together on a joint project to merge existing technology (the iPhone) with a real-world crowd sourcing application to help blind people make ...

Crowd-talk yields great answers, says university team

September 10, 2012

(—Move over, Siri. Some researchers from the University of Rochester in collaboration with a University of California, Berkeley, mathematician/crowdsourcing entrepreneur, have come up with a killer personal assistant ...

Research helps businesses optimize benefits of crowdsourcing

November 28, 2012

(—In today's global marketplace, a growing number of organizations are attempting to gain an edge over their competitors through "crowdsourcing"—the use of large groups of individuals to perform tasks commonly ...

Recommended for you

Cryptocurrency rivals snap at Bitcoin's heels

January 14, 2018

Bitcoin may be the most famous cryptocurrency but, despite a dizzying rise, it's not the most lucrative one and far from alone in a universe that counts 1,400 rivals, and counting.

Top takeaways from Consumers Electronics Show

January 13, 2018

The 2018 Consumer Electronics Show, which concluded Friday in Las Vegas, drew some 4,000 exhibitors from dozens of countries and more than 170,000 attendees, showcased some of the latest from the technology world.

Finnish firm detects new Intel security flaw

January 12, 2018

A new security flaw has been found in Intel hardware which could enable hackers to access corporate laptops remotely, Finnish cybersecurity specialist F-Secure said on Friday.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.