Decision making in scientific peer review unravelled with mathematical modelling

April 22, 2013

Professor Mikko Alava and undergraduate Tuomo Hartonen of Aalto University Department of Applied Physics have modelled the work processes and human decision making in scientific peer review with the help of statistical physics. Their study will improve understanding of how actions of reviewers and editors during the review work correlate with the decisions to publish or reject article manuscripts. The article presenting their research is now published in the journal Scientific Reports.

Peer review is one of the most pivotal guarantees of advancement in and its objectivity. In practice, it comprises sending article manuscripts to journals, extensive commentary reading, corrections and suggestions, correspondence and decision making from scientists, reviewers and editors alike.

In their Nature's Scientific Reports article, Alava and Hartonen show that the time reviewers spend on working on a manuscript is clearly dependant on the perceived quality of the manuscript and on the decision whether to publish or to reject it in the end.

Alava and Hartonen study peer review as a series of dynamic processes, in which the reviewers' work phases and the decision making that comes with it must be modelled simultaneously. They have studied the of over 10 000 manuscripts in two physics journals all the way from submission to the final verdict of publication or rejection.

The work patterns and decision making are included in a that takes heed of the varying tasks during the review process. Review work is always occurs in circumstances, in which many concomitant tasks and priorities mix. Alava and Hartonen call this the Deadline-effect: important tasks are completed amidst a constant friction caused by competitive tasks as a time limit approaches. Here decision making must be modelled as stochastic processes, that is, to mathematically manage the unpredictable of task completion.

The amount of time that reviewers and editors take to process submitted articles depends on the perceived quality of the manuscripts. Prominent and poor manuscripts take a lot less time and work phases to go through peer review than the average majority. A manuscript that seems good at first glance features in the data, which Alava and Hartonen have analysed, as a stream of decisions markedly unaffected by the Deadline-effect and pressure.

Final verdicts made on manuscripts still define the statistics of processes to such an extent that their integrity remains intact in the study. Alava and Hartonen conclude that expert reviewers make their decisions based on educated and informed guesses rather than on subjectively biased judgments.

Explore further: To publish or not to publish? That is the question

More information: Read Tuomo Hartonen's and Mikko Alava's article 'How important tasks are performed: peer review' online.

Related Stories

To publish or not to publish? That is the question

May 21, 2010

For more than 50 years medical research has been vetted through the peer-review process overseen by medical journal editors who assign reviewers to determine whether work merits publication. A study published in PLoS One ...

Online game aims to improve scientific peer review accuracy

November 9, 2011

Peer review of scientific research is an essential component of research publication, the awarding of grants, and academic promotion. Reviewers are often anonymous. However, a new study by researchers at the Johns Hopkins ...

Federal peer review may be overstretched and error prone

January 10, 2011

The federal peer review system, by which research proposals are judged worthy for funding, may be "over stretched" and "susceptible to error," said Elmer Yglesias, a researcher at the Science and Technology Policy Institute ...

Recommended for you

Industrial "edge cities" have helped China grow

August 18, 2017

China's massive investment in industrial parks has paid economic dividends while reshaping the urban areas where they are located, according to a newly published study co-authored by an MIT expert on urban economics.

Ancient species of giant sloth discovered in Mexico

August 17, 2017

Mexican scientists said Wednesday they have discovered the fossilized remains of a previously unknown species of giant sloth that lived 10,000 years ago and died at the bottom of a sinkhole.

The mathematics of golf

August 16, 2017

(—The official Rules of Golf, which are continually being revised and updated as new equipment emerges, have close ties to mathematics. In many cases, math is used to place limitations on golf equipment, such as ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Apr 22, 2013
Alava and Hartonen show that the time reviewers spend on working on a manuscript is clearly dependant on the perceived quality of the manuscript

Reason being: Peer review takes time. You don't get paid for it (i.e. as a reviewer you're actively spending money on it with every second you spend reviewing a paper).

Of course will you spend less time on a paper which, after first reading, seems marginally important as opposed to one that seems to contain genuinely innovative ideas.

Also when you get a paper for review you usually don't do it right away. Mostly you have other work to do as a researcher (more than you can fit into your work hours, assuredly). So review is one of those things that you put off 'until you have the time' (which usually means until a few days or the day before deadline).

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.