Decision making in scientific peer review unravelled with mathematical modelling

Apr 22, 2013

Professor Mikko Alava and undergraduate Tuomo Hartonen of Aalto University Department of Applied Physics have modelled the work processes and human decision making in scientific peer review with the help of statistical physics. Their study will improve understanding of how actions of reviewers and editors during the review work correlate with the decisions to publish or reject article manuscripts. The article presenting their research is now published in the journal Scientific Reports.

Peer review is one of the most pivotal guarantees of advancement in and its objectivity. In practice, it comprises sending article manuscripts to journals, extensive commentary reading, corrections and suggestions, correspondence and decision making from scientists, reviewers and editors alike.

In their Nature's Scientific Reports article, Alava and Hartonen show that the time reviewers spend on working on a manuscript is clearly dependant on the perceived quality of the manuscript and on the decision whether to publish or to reject it in the end.

Alava and Hartonen study peer review as a series of dynamic processes, in which the reviewers' work phases and the decision making that comes with it must be modelled simultaneously. They have studied the of over 10 000 manuscripts in two physics journals all the way from submission to the final verdict of publication or rejection.

The work patterns and decision making are included in a that takes heed of the varying tasks during the review process. Review work is always occurs in circumstances, in which many concomitant tasks and priorities mix. Alava and Hartonen call this the Deadline-effect: important tasks are completed amidst a constant friction caused by competitive tasks as a time limit approaches. Here decision making must be modelled as stochastic processes, that is, to mathematically manage the unpredictable of task completion.

The amount of time that reviewers and editors take to process submitted articles depends on the perceived quality of the manuscripts. Prominent and poor manuscripts take a lot less time and work phases to go through peer review than the average majority. A manuscript that seems good at first glance features in the data, which Alava and Hartonen have analysed, as a stream of decisions markedly unaffected by the Deadline-effect and pressure.

Final verdicts made on manuscripts still define the statistics of processes to such an extent that their integrity remains intact in the study. Alava and Hartonen conclude that expert reviewers make their decisions based on educated and informed guesses rather than on subjectively biased judgments.

Explore further: Researchers help Boston Marathon organizers plan for 2014 race

More information: Read Tuomo Hartonen's and Mikko Alava's article 'How important tasks are performed: peer review' online.

add to favorites email to friend print save as pdf

Related Stories

To publish or not to publish? That is the question

May 21, 2010

For more than 50 years medical research has been vetted through the peer-review process overseen by medical journal editors who assign reviewers to determine whether work merits publication. A study published in PLoS One invest ...

Online game aims to improve scientific peer review accuracy

Nov 09, 2011

Peer review of scientific research is an essential component of research publication, the awarding of grants, and academic promotion. Reviewers are often anonymous. However, a new study by researchers at the Johns Hopkins ...

Federal peer review may be overstretched and error prone

Jan 10, 2011

The federal peer review system, by which research proposals are judged worthy for funding, may be "over stretched" and "susceptible to error," said Elmer Yglesias, a researcher at the Science and Technology Policy Institute ...

Recommended for you

Egypt archaeologists find ancient writer's tomb

3 hours ago

Egypt's minister of antiquities says a team of Spanish archaeologists has discovered two tombs in the southern part of the country, one of them belonging to a writer and containing a trove of artifacts including reed pens ...

Study finds law dramatically curbing need for speed

Apr 18, 2014

Almost seven years have passed since Ontario's street-racing legislation hit the books and, according to one Western researcher, it has succeeded in putting the brakes on the number of convictions and, more importantly, injuries ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

antialias_physorg
not rated yet Apr 22, 2013
Alava and Hartonen show that the time reviewers spend on working on a manuscript is clearly dependant on the perceived quality of the manuscript

Reason being: Peer review takes time. You don't get paid for it (i.e. as a reviewer you're actively spending money on it with every second you spend reviewing a paper).

Of course will you spend less time on a paper which, after first reading, seems marginally important as opposed to one that seems to contain genuinely innovative ideas.

Also when you get a paper for review you usually don't do it right away. Mostly you have other work to do as a researcher (more than you can fit into your work hours, assuredly). So review is one of those things that you put off 'until you have the time' (which usually means until a few days or the day before deadline).

More news stories

Egypt archaeologists find ancient writer's tomb

Egypt's minister of antiquities says a team of Spanish archaeologists has discovered two tombs in the southern part of the country, one of them belonging to a writer and containing a trove of artifacts including reed pens ...

NASA's space station Robonaut finally getting legs

Robonaut, the first out-of-this-world humanoid, is finally getting its space legs. For three years, Robonaut has had to manage from the waist up. This new pair of legs means the experimental robot—now stuck ...

Ex-Apple chief plans mobile phone for India

Former Apple chief executive John Sculley, whose marketing skills helped bring the personal computer to desktops worldwide, says he plans to launch a mobile phone in India to exploit its still largely untapped ...

Filipino tests negative for Middle East virus

A Filipino nurse who tested positive for the Middle East virus has been found free of infection in a subsequent examination after he returned home, Philippine health officials said Saturday.

Airbnb rental site raises $450 mn

Online lodging listings website Airbnb inked a $450 million funding deal with investors led by TPG, a source close to the matter said Friday.