Journal study suggests forcing open peer review could lead to more bias

An entomologist at the University of Kentucky has found that requiring peer review reviewers to sign their names to their review comments could bias their results. In his paper published in Proceedings of the Royal Society B: Biological Sciences, Charles Fox, describe how he studied peer-reviewed articles for one prominent journal and what he learned about the process.

When a researcher wants a journal to publish their research paper, they submit it to an editor who then passes it on to several experts in the field for review. Normally, the reviewers are given the option of signing their review after they complete it but few do so, preferring to keep their remarks anonymous. Logic suggests that most do so to prevent the research team from retaliating for negative comments. In this new effort, Fox noted that as part of the open publishing push by some in the science community, some have suggested that all reviewers be required to reveal their identities—he wondered what impact that might have on not only the person doing the review but the paper under review.

To learn more, he examined the reviews for papers accepted by the journal Functional Ecology for the years 2003 to 2005 and for 2013 through 2015. In looking at his data, he found that just 5.6% of reviewers signed their comments. He also found that male reviewers were twice as likely to sign their reviews as women and that reviewers were much more likely to reveal their identity if they left mostly positive reviews. And he found that reviewers who had been suggested by authors of the paper were more likely to put their name on their reviews as well. Also, interestingly, he found that reviewers who chose the option 'Professor' as their salutation, were 1.6 times as likely to sign their names as reviewers who chose to label themselves as simply 'Dr.'

Wolf suggests the hesitancy demonstrated by reviewers suggests they prefer to remain anonymous. He further suggests that the data indicates that requiring reviewers to sign their names could introduce bias into the comments that are made. And finally, he suggests that the hesitancy appeared to be more pronounced for both female and junior reviewers.