Researchers find malicious behavior the norm in crowdsourcing contests

Aug 20, 2014 by Bob Yirka report
Credit: George Hodan/Public Domain

(Phys.org) —A small team of researchers with members from facilities in the U.K., Australia, and Saudi Arabia has found that when it comes to crowdsourcing contests, malicious behavior is the norm. In their paper published in Journal of the Royal Society Interface, the researchers describe how they applied game theory to data collected from crowdsourced contests and what their analysis revealed.

With the advent of the Internet, has become a popular way to get things done—projects born on sites such as Kickstarter benefit from many small investors, for example. Crowdsourcing has also led to that are able to quickly utilize the talents of many people across the globe to solve problems that would take a small team much more time. Because of that, some groups have begun studying crowdsourcing itself to learn more about how it actually works in the real world. In this latest effort, the researchers focused on crowdsourcing contests.

Crowdsourcing contests are held for a variety of reasons; because they can be fun, because someone or a group wants to learn something, or because it seems the only way to solve a problem. But one of the main strengths of crowdsourcing is also one of its main weaknesses—the necessary openness that allows anyone that wishes to do so to compete. That openness also allows others to bend the rules to give them an advantage in a competition, or to disrupt the contest for whatever reasons they may harbor. To learn more about how vulnerable such contests may be, the researchers gathered data from several recent well known crowdsourcing contests, such as the one sponsored by DARPA to find balloons that had been hidden.

In studying the data, they found that malicious behavior was the norm, rather than the exception—which they believe runs contrary to popular perceptions. Sadly, they found that making it more difficult for people to conduct malicious behavior during a contest didn't really stop it from happening and instead led to a less desirable outcome in most cases. This means, they conclude, that caution should be used by those planning such contests while researchers continue looking into ways to prevent from ruining outcomes.

Explore further: NIH turns to crowdsourcing to repurpose drugs

More information: Crowdsourcing contest dilemma, J. R. Soc. Interface 6 October 2014 vol. 11 no. 99 20140532. Published 20 August 2014 DOI: 10.1098/rsif.2014.0532

Abstract
Crowdsourcing offers unprecedented potential for solving tasks efficiently by tapping into the skills of large groups of people. A salient feature of crowdsourcing—its openness of entry—makes it vulnerable to malicious behaviour. Such behaviour took place in a number of recent popular crowdsourcing competitions. We provide game-theoretic analysis of a fundamental trade-off between the potential for increased productivity and the possibility of being set back by malicious behaviour. Our results show that in crowdsourcing competitions malicious behaviour is the norm, not the anomaly—a result contrary to the conventional wisdom in the area. Counterintuitively, making the attacks more costly does not deter them but leads to a less desirable outcome. These findings have cautionary implications for the design of crowdsourcing competitions.

add to favorites email to friend print save as pdf

Related Stories

NIH turns to crowdsourcing to repurpose drugs

Jul 16, 2014

Experimental drugs proven safe but perhaps not sufficiently effective in initial testing or against a first disease target may sit gathering dust on the shelves of pharmaceutical companies. An NIH-sponsored effort based on ...

Crowdsourcing may help dieters lose weight

Aug 04, 2014

Crowdsourcing may help dieters stick to healthy foods and lose weight, as participants are as good as trained experts at correctly rating the healthiness of foods and giving feedback on them, indicates research published ...

Crowdsourcing the phase problem

Jun 17, 2014

Compared with humans, computers have the capacity to solve problems at much greater speed. There are many problems, however, where computational speed alone is insufficient to find a correct or optimal solution, ...

Millions join satellite search for missing plane

Mar 18, 2014

Three million people have joined an effort led by a satellite operator to locate the missing Malaysia Airlines plane, in what may be the largest crowdsourcing project of its kind.

Recommended for you

World population likely to peak by 2070

Oct 23, 2014

World population will likely peak at around 9.4 billion around 2070 and then decline to around 9 billion by 2100, according to new population projections from IIASA researchers, published in a new book, World Population and ...

Bullying in schools is still prevalent, national report says

Oct 23, 2014

Despite a dramatic increase in public awareness and anti-bullying legislation nationwide, the prevalence of bullying is still one of the most pressing issues facing our nation's youth, according to a report by researchers ...

Study examines effects of credentialing, personalization

Oct 23, 2014

Chris Gamrat, a doctoral student in learning, design and technology, recently had his study—completed alongside Heather Zimmerman, associate professor of education; Jaclyn Dudek, a doctoral student studying learning, design ...

User comments : 8

Adjust slider to filter visible comments by rank

Display comments: newest first

julianpenrod
1.4 / 5 (9) Aug 20, 2014
If we assume that vicious slugs who love to see the pristine desecrated, the whole destroyed are not the majority in society in general, then this suggests what many have suspected for a long time. That, in the societal sub niche of the "trendy", the followers, not the leaders, tend to be maliginant and hate filled. One need only look at comments on blogs on everything from God to "science" to politics to see this kind of behavior. Lice infested, arrested behavior losers stealing money from their mothers' purses or beating their mothers up for money. Even here, a collection of malingerers wait until I place a comment then dutifully give me "1's" to dilute any higher ranking I might get. They don't contest me statements because they know I am right, they do it only to be vicious. Yes, a number may be New World Order moles, but that doesn't necessarily make them less craven.
Shakescene21
1.9 / 5 (8) Aug 20, 2014
@julienpenrod -- I ignore the rankings because they are so often distorted by "malicious behavior". You should too, rather than be concerned about it.
Several of us have suggested that Physorg should discontinue this abused feature, and maybe the Physorg editors will actually read this study. The Physorg rankings are a good example of the behavior that the researchers have documented.
Vietvet
4.4 / 5 (8) Aug 20, 2014
This study could be applied to the science denying and nasty trolls like julianpenrod.
supamark23
4.6 / 5 (9) Aug 20, 2014
@julienpenrod - you get downvoted because you say stupid things. The feature exists, in part, because phys.org doesn't moderate their forums.
TheGhostofOtto1923
3.4 / 5 (5) Aug 20, 2014
Lice infested, arrested behavior losers stealing money from their mothers' purses or beating their mothers up for money
Well I guess it all depends on what adjectives you use.

"34... I have not come to bring peace, but a sword. 35 For I have come to set a man against his father, and a daughter against her mother, and a daughter-in-law against her mother-in-law." Mat10

""If anyone comes to me and does not hate father and mother, wife and children, brothers and sisters—yes, even their own life—such a person cannot be my disciple." Luk14

-The bible condones all these things and more in the proper context, yes? Are they feeding their drug addiction or their jesus addiction? The tithe is mandatory you know.
Even here, a collection of malingerers wait until I place a comment then dutifully give me "1's" to dilute any higher ranking I might get
But you dont get anything BUT 1s julien. Havent you noticed?

Oh thats right - evidence gives you the heebie jeebies. I understand.
TheGhostofOtto1923
3.4 / 5 (5) Aug 20, 2014
If we assume that vicious slugs who love to see the pristine desecrated, the whole destroyed are not the majority in society in general
Well we can assume that this is how you regard all those who do not believe in your personal god. And unfortunately for you, they ARE indeed in the majority.
then this suggests what many have suspected for a long time. That, in the societal sub niche of the "trendy", the followers, not the leaders, tend to be maliginant and hate filled
Nah its godders like you who regard unbelievers as
vicious slugs
and
Lice infested, arrested behavior losers stealing money from their mothers' purses or beating their mothers up for money
who are malignant and hate-filled. Thanks for exposing the true nature of your faith for us.
Toiea
2 / 5 (4) Aug 20, 2014
you get downvoted because you say stupid things. The feature exists, in part, because phys.org doesn't moderate their forums
I dunno what's silly about it - the abusing of public voting systems, encyclopedia, grants, etc. is just a bare fact. The fact these features exist for some reason doesn't contradict this view. I'd say, what is stupid here is rather the comment of yours due to presence of this apparent logical fallacy.
Jeweller
5 / 5 (3) Aug 20, 2014
Here's my two cents worth, from Brian in Cape Town.
Anonymity seems to bring out the worst behaviour in a lot of people because it allows them to avoid facing the consequences of their words or actions.
Create traceability and as a result, you will have accountability.
That will greatly reduce maliciousness.

These are the words of the..........
Retired Old Jeweller
Brian