August 20, 2014 report
Researchers find malicious behavior the norm in crowdsourcing contests
(Phys.org) —A small team of researchers with members from facilities in the U.K., Australia, and Saudi Arabia has found that when it comes to crowdsourcing contests, malicious behavior is the norm. In their paper published in Journal of the Royal Society Interface, the researchers describe how they applied game theory to data collected from crowdsourced contests and what their analysis revealed.
With the advent of the Internet, crowdsourcing has become a popular way to get things done—projects born on sites such as Kickstarter benefit from many small investors, for example. Crowdsourcing has also led to research projects that are able to quickly utilize the talents of many people across the globe to solve problems that would take a small team much more time. Because of that, some groups have begun studying crowdsourcing itself to learn more about how it actually works in the real world. In this latest effort, the researchers focused on crowdsourcing contests.
Crowdsourcing contests are held for a variety of reasons; because they can be fun, because someone or a group wants to learn something, or because it seems the only way to solve a problem. But one of the main strengths of crowdsourcing is also one of its main weaknesses—the necessary openness that allows anyone that wishes to do so to compete. That openness also allows others to bend the rules to give them an advantage in a competition, or to disrupt the contest for whatever reasons they may harbor. To learn more about how vulnerable such contests may be, the researchers gathered data from several recent well known crowdsourcing contests, such as the one sponsored by DARPA to find balloons that had been hidden.
In studying the data, they found that malicious behavior was the norm, rather than the exception—which they believe runs contrary to popular perceptions. Sadly, they found that making it more difficult for people to conduct malicious behavior during a contest didn't really stop it from happening and instead led to a less desirable outcome in most cases. This means, they conclude, that caution should be used by those planning such contests while researchers continue looking into ways to prevent malicious behavior from ruining outcomes.
More information: Crowdsourcing contest dilemma, J. R. Soc. Interface 6 October 2014 vol. 11 no. 99 20140532. Published 20 August 2014 DOI: 10.1098/rsif.2014.0532
Abstract
Crowdsourcing offers unprecedented potential for solving tasks efficiently by tapping into the skills of large groups of people. A salient feature of crowdsourcing—its openness of entry—makes it vulnerable to malicious behaviour. Such behaviour took place in a number of recent popular crowdsourcing competitions. We provide game-theoretic analysis of a fundamental trade-off between the potential for increased productivity and the possibility of being set back by malicious behaviour. Our results show that in crowdsourcing competitions malicious behaviour is the norm, not the anomaly—a result contrary to the conventional wisdom in the area. Counterintuitively, making the attacks more costly does not deter them but leads to a less desirable outcome. These findings have cautionary implications for the design of crowdsourcing competitions.
Journal information: Journal of the Royal Society Interface
© 2014 Phys.org
