BES launches large-scale study to test whether 'blinding' reduces bias in science publishing

Scientific papers go through a peer-review process before they are accepted for publication in a journal. They are sent to two or more independent researchers for comment. Those researchers are asked to assess the robustness of the methods used and the conclusions drawn, as well as the novelty of the study. The reviewers' comments play an important role in determining which papers get accepted and published.

Currently the most common process in ecology journals is single blind peer review. The authors of a submitted research paper are not told who the reviewers are: they are 'blind' to their identity. The reviewers, however, do see the authors' details on the papers they check.

This has led to concerns that, consciously or unconsciously, knowledge of an author's gender, university position or nationality could influence how reviewers assess the research reported in the paper.

For example, there is some mixed evidence to suggest research led by female authors may not score as highly in peer review. On the other hand, papers by well-known authors or those from prestigious institutions may get an easier ride.

As a result, some journals have introduced 'double-blinding'. This is where the reviewers are 'blinded' as well: the list of authors is removed from the paper they look at.

Beginning 5 September, the British Ecological Society will conduct a large-scale randomised controlled trial over the next two years to compare the two approaches to peer review. An estimated 2500 research papers submitted to the journal Functional Ecology will be randomly allotted into two workflows: single- or double-blind peer review. The study will assess whether author characteristics affect peer-review scores and acceptance of papers, as well as the effectiveness of the blinding process.

In particular, the study will examine whether double-blinding reduces variation in peer review scores and acceptance rates among authors of different genders, geographic locations, first languages (English vs. other), university prestige, career stages (junior vs. senior) and publishing histories (more prolific, higher prestige).

The journal will also investigate how anonymising authors influences the publishing process—in its ability to recruit reviewers, the quality of reviews received, the average rating given to papers and the ability of reviewers to identify authors. Finally, authors and reviewers will be asked to complete a survey of their opinions on single- and double-blind peer review.

Charles Fox, Executive Editor for Functional Ecology, said: "It's critical for science, and for the scientists involved, that the research which gets published is selected through a fair and unbiased process. We know that people are concerned bias in peer review can act against female researchers and those from developing countries. As scientists, we should seek to base our processes on the best evidence. That's why we're carrying out this trial. The results will help determine the best ways of minimising sources of potential bias in the publishing process."

Catherine Hill, Director of Publishing at the British Ecological Society, said: "The British Ecological Society is committed to improving openness and inclusivity in our science. We have six leading journals publishing the latest in ecology. It's vitally important that the research we publish is reviewed and selected in the most impartial way, regardless of the authors' backgrounds. This study will provide important data on whether we are achieving this aim and ensure our peer review policy is based on the best possible evidence."