Research pair suggest crowd sourcing could improve social science experiment results

October 8, 2015 by Bob Yirka, Phys.org report
Credit: George Hodan/Public Domain

(Phys.org)—Social science experiments do not have the best track records—oftentimes, studies are conducted, followed by wild claims which are then either contested or outright refuted by colleagues. There might be a better way Raphael Silberzahn and Eric Uhlmann suggest in an essay piece published in the journal Nature—crowdsourcing.

The two social scientists have experience in this area, together they published the results of a study they conducted a couple of years ago where they found that German people with royal sounding last names, tended to become business managers. A colleague asked for their data and after conducting an analysis of his own, subsequently refuted the claims made by the pair. This led the pair and some of their colleagues to look for a way to improve social science experiment results—they came up with crowd sourcing.

To test their idea, the researchers picked a topic—do soccer refs give red cards to dark skinned people more often than light skinned people? To find out, they consulted with a sports data gathering company that pulled statistics from actual matches—the data was then given to 29 teams of researchers who volunteered to give their time to the project. Each team was asked to use the same data given to them to make a determination, with no other constraints. Each team sent back their results along with the methods they used to obtain them. Each of the research team was then sent a document outlining all the methodologies used by all the teams, but without the results and were asked to rate them and to provide feedback. Each research team received all feedback and ratings and a chance to change their initial findings. Then, finally, all of the teams were invited to participate in an e-mail exchange to discus methodologies and results. Teams were then given another chance to modify their own results and then a final document was drawn up that gave a group consensus on the original question.

The result was that 20 of the research found that darker skin color did indeed lead to more red cards, but the researchers varied widely on how prevalent it was—the group consensus was that it was 1.3 times more likely. And while that number is significant all by itself, what is really important Silberzahn and Uhlmann write, is that the experiment revealed that using crowd sourcing resulted in a more hesitant response from researchers, which they believe led to a reduced incentive for publishing flashy results.

Explore further: Collaboration study shows people lie more when colluding

More information: Crowdsourced research: Many hands make tight work, Nature 526, 189–191 (08 October 2015) DOI: 10.1038/526189a , http://www.nature.com/news/crowdsourced-research-many-hands-make-tight-work-1.18508

Related Stories

Collaboration study shows people lie more when colluding

August 11, 2015

(Phys.org)—A pair of researchers, one with the University of Nottingham in the U.K., the other with Ben-Gurion University of the Negev in Israel, has carried out a study on the downside of collaborative efforts, and has ...

Study shows increased aggression between evenly matched teams

August 16, 2012

(Phys.org) -- Over the years, research has shown that when two people, or animals, are evenly matched opponents, both tend to resort to more savagery to win than if one is clearly superior to the other. Now new research by ...

Study tests motivational techniques

January 22, 2015

Every year, U.S. companies spend billions on incentives for salespeople, and although incentives can help boost sales figures, the associated costs cut into the bottom line in a big way.

Recommended for you

In the gaping mouth of ancient crocodiles

June 18, 2018

The mouth of today's crocodilians inspires fear and awe, with their wide gape and the greatest known bite force in the vertebrate animal kingdom. However, this apex predator of today and its modus of attack (its mouth) had ...

Secrets of extinct cow with face like a bulldog revealed

June 15, 2018

An international team of scientists have used the latest genetic and anatomical techniques to study the remains of a cow with a short face like a bulldog that fascinated Charles Darwin when he first saw it in Argentina 180 ...

Ancient mammal ancestor found and identified in China

June 14, 2018

A team of researchers from China and the U.S. has identified a new ancient mammal ancestor recently found in a part of China. In their paper published in the journal Nature, the group describes the creature, which they have ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.