Online game aims to improve scientific peer review accuracy

November 9, 2011

Peer review of scientific research is an essential component of research publication, the awarding of grants, and academic promotion. Reviewers are often anonymous. However, a new study by researchers at the Johns Hopkins Bloomberg School of Public Health found that greater cooperation between reviewer and author can improve accuracy of the review. Their study is published in the Nov. 9 edition of the journal PLoS ONE.

To examine the accuracy of different review processes, the Johns Hopkins researchers developed a model using an online game on the Amazon E2 cloud. Participants were asked to solve and review questions from the GRE (Graduate Record Examinations). The study examined both closed review, in which the author did not know the , and open review, where the author knows the reviewers.

The study found that when review behavior was public and under open review, cooperative interactions increased 13 percent. Overall accuracy between closed and open review models was similar. However, reviewers and authors who participated in cooperative interactions had an 11 percent higher reviewing accuracy rate.

"Our results suggest that increasing cooperation in the process could reduce the risk of reviewing errors," said Jeffrey Leek, PhD, lead author of the study and assistant professor in the Bloomberg School's Department of Biostatistics.

Explore further: A re-review of peer review: Leading journal looks to end the 'review nightmare'

More information: Leek JT, Taub MA, Pineda FJ (2011) Cooperation between Referees and Authors Increases Peer Review Accuracy. PLoS ONE 6(11): e26895. doi:10.1371/journal.pone.0026895

Related Stories

Peer Review Survey 2009: Preliminary findings

September 8, 2009

Should peer review detect fraud and misconduct? What does it do for science and what does the scientific community want it to do? Will it illuminate good ideas or shut them down? Should reviewers remain anonymous?

To publish or not to publish? That is the question

May 21, 2010

For more than 50 years medical research has been vetted through the peer-review process overseen by medical journal editors who assign reviewers to determine whether work merits publication. A study published in PLoS One ...

Federal peer review may be overstretched and error prone

January 10, 2011

The federal peer review system, by which research proposals are judged worthy for funding, may be "over stretched" and "susceptible to error," said Elmer Yglesias, a researcher at the Science and Technology Policy Institute ...

Amazon's elite product reviewers

June 15, 2011

(PhysOrg.com) -- Comments about a product online can make or break a sale. But who are the people behind the reviews and why do they do it?

Recommended for you

French teen finds 560,000 year-old tooth (Update)

July 28, 2015

A 16-year-old French volunteer archaeologist has found an adult tooth dating back around 560,000 years in southwestern France, in what researchers hailed as a "major discovery" Tuesday.

The couple who Facebooks together, stays together

July 27, 2015

Becoming "Facebook official" is a milestone in modern romance, and new research suggests that activities on the popular social networking site are connected to whether those relationships last.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.