Online game aims to improve scientific peer review accuracy

Nov 09, 2011

Peer review of scientific research is an essential component of research publication, the awarding of grants, and academic promotion. Reviewers are often anonymous. However, a new study by researchers at the Johns Hopkins Bloomberg School of Public Health found that greater cooperation between reviewer and author can improve accuracy of the review. Their study is published in the Nov. 9 edition of the journal PLoS ONE.

To examine the accuracy of different review processes, the Johns Hopkins researchers developed a model using an online game on the Amazon E2 cloud. Participants were asked to solve and review questions from the GRE (Graduate Record Examinations). The study examined both closed review, in which the author did not know the , and open review, where the author knows the reviewers.

The study found that when review behavior was public and under open review, cooperative interactions increased 13 percent. Overall accuracy between closed and open review models was similar. However, reviewers and authors who participated in cooperative interactions had an 11 percent higher reviewing accuracy rate.

"Our results suggest that increasing cooperation in the process could reduce the risk of reviewing errors," said Jeffrey Leek, PhD, lead author of the study and assistant professor in the Bloomberg School's Department of Biostatistics.

Explore further: Best of Last Week – New type of qubit created, Hubble sees a glowing galaxy and extreme agreeing may solve disagreements

More information: Leek JT, Taub MA, Pineda FJ (2011) Cooperation between Referees and Authors Increases Peer Review Accuracy. PLoS ONE 6(11): e26895. doi:10.1371/journal.pone.0026895

add to favorites email to friend print save as pdf

Related Stories

Federal peer review may be overstretched and error prone

Jan 10, 2011

The federal peer review system, by which research proposals are judged worthy for funding, may be "over stretched" and "susceptible to error," said Elmer Yglesias, a researcher at the Science and Technology Policy Institute ...

To publish or not to publish? That is the question

May 21, 2010

For more than 50 years medical research has been vetted through the peer-review process overseen by medical journal editors who assign reviewers to determine whether work merits publication. A study published in PLoS One invest ...

Peer Review Survey 2009: Preliminary findings

Sep 08, 2009

Should peer review detect fraud and misconduct? What does it do for science and what does the scientific community want it to do? Will it illuminate good ideas or shut them down? Should reviewers remain anonymous?

Amazon's elite product reviewers

Jun 15, 2011

(PhysOrg.com) -- Comments about a product online can make or break a sale. But who are the people behind the reviews and why do they do it?

Recommended for you

How to win a Tour de France sprint

11 hours ago

The final dash to the line in a Tour de France sprint finish may appear to the bystander to be a mess of bodies trying to cram into the width of a road, but there is a high degree of strategy involved. It ...

Bible museum planned for US capital

Jul 18, 2014

The devout Christian family that upended a part of President Barack Obama's health care law aims to open a Bible museum in Washington in 2017, a spokesperson for the project said Friday.

The science behind Tour de France's hide-and-seek tactics

Jul 15, 2014

When the Tour de France comes to town, it's a chance to get your gladrags on. This year's Grand Depart in Yorkshire saw Leeds decked out with yellow flowers, bikes placed in coffee bar windows, statues wearing ...

User comments : 0