Online game aims to improve scientific peer review accuracy

Nov 09, 2011

Peer review of scientific research is an essential component of research publication, the awarding of grants, and academic promotion. Reviewers are often anonymous. However, a new study by researchers at the Johns Hopkins Bloomberg School of Public Health found that greater cooperation between reviewer and author can improve accuracy of the review. Their study is published in the Nov. 9 edition of the journal PLoS ONE.

To examine the accuracy of different review processes, the Johns Hopkins researchers developed a model using an online game on the Amazon E2 cloud. Participants were asked to solve and review questions from the GRE (Graduate Record Examinations). The study examined both closed review, in which the author did not know the , and open review, where the author knows the reviewers.

The study found that when review behavior was public and under open review, cooperative interactions increased 13 percent. Overall accuracy between closed and open review models was similar. However, reviewers and authors who participated in cooperative interactions had an 11 percent higher reviewing accuracy rate.

"Our results suggest that increasing cooperation in the process could reduce the risk of reviewing errors," said Jeffrey Leek, PhD, lead author of the study and assistant professor in the Bloomberg School's Department of Biostatistics.

Explore further: New anthology offers comprehensive insight into the life and works of Margaret Thatcher

More information: Leek JT, Taub MA, Pineda FJ (2011) Cooperation between Referees and Authors Increases Peer Review Accuracy. PLoS ONE 6(11): e26895. doi:10.1371/journal.pone.0026895

add to favorites email to friend print save as pdf

Related Stories

Federal peer review may be overstretched and error prone

Jan 10, 2011

The federal peer review system, by which research proposals are judged worthy for funding, may be "over stretched" and "susceptible to error," said Elmer Yglesias, a researcher at the Science and Technology Policy Institute ...

To publish or not to publish? That is the question

May 21, 2010

For more than 50 years medical research has been vetted through the peer-review process overseen by medical journal editors who assign reviewers to determine whether work merits publication. A study published in PLoS One invest ...

Peer Review Survey 2009: Preliminary findings

Sep 08, 2009

Should peer review detect fraud and misconduct? What does it do for science and what does the scientific community want it to do? Will it illuminate good ideas or shut them down? Should reviewers remain anonymous?

Amazon's elite product reviewers

Jun 15, 2011

(PhysOrg.com) -- Comments about a product online can make or break a sale. But who are the people behind the reviews and why do they do it?

Recommended for you

James Watson's Nobel Prize to be auctioned

Nov 25, 2014

Missed the chance to bid on Francis Crick's Nobel Prize when it was auctioned off last year for $2.27 million? No worries, you'll have another chance to own a piece of science history on Dec. 4, when James D. Watson's 1962 ...

Engineers develop gift guide for parents

Nov 21, 2014

Faculty and staff in Purdue University's College of Engineering have come up with a holiday gift guide that can help engage children in engineering concepts.

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.