Expertise in sciences and the decision of what is publishable

When Einstein discovered the peer-review process…

In 1935, a too-quick calculation led Einstein to believe that gravitation waves could not exist in the frame of the general relativity theory (these waves, observed in 2016 only, are actually a cornerstone of his work). The story of his initial publication is richer than the subtle error behind it…

For the first time, Einstein – who had sent his manuscript to the prestigious Physical Review – faced the anonymous peer-review system. The sharp-minded reviewer, whose identity was revealed only in 2005, pointed out an error. Einstein strongly disagreed with the idea that an editor could review his work without his consent. He then sent his manuscript to another review, which decided to publish it. But when the time came to check the manuscript's proofs, Einstein totally revised his paper. This story, both exemplary and exceptional, illustrates the complex relationship between the scientists and the publications.

The daily life of expertise in scientific publications

The German journals in which Einstein published had a low rejection rate of submitted materials, and were instead open to controversies and scientific debates. Nevertheless, the immense growth of scientific activity forced all scientific journals to follow the example of Physical Review, with one or two (even three sometimes) anonymous reviewers.

Rather than the editor, who's responsible for the selection of the publications, the reviewers are those who judge whether the work is valid and deserves the (rather subjective) recognition attached to publication in a given review. Further criticism remains possible: comments to papers can be published after approval by a reviewing process, possibly followed by an author reply.

Once scientists are contacted as possible reviewers by the editor, they have only a short period of time to accept. They will have few weeks – sometimes even less – to deliver informed opinions on the suitability of the manuscript. This unpaid work remains anonymous (with some exceptions).

From my own experience, the time required can vary from one hour to three days. The noble and exciting part of reviewing is that the process of critical reading and author replies sometimes creates a kind of "co-production" for the content that finally appears in the journal: The reviewer often helps improve the paper's readability and presentation, and sometimes the reviewer suggests broader openings not mentioned by the authors. The reviewer can also discover small or even serious errors, though generally they're not critical for the work's conclusion, which is thus free of errors when finally published.

Finding good reviewers: the tough part of an editor's job

Finding available reviewers is a difficult task for journal editors. Specialists with both the required expertise and a sufficiently large view of the field are rare. They are busy people, and often prefer to review manuscripts when they introduce new ideas rather than evaluating the correctness of a merely incremental paper. Early-career scientists are often more open, as they can enjoy participating in the peer-reviewing process, which is at the heart of the academic system they want to join.

One problem is the growth in the number of submitted manuscripts, which in turn require more and more reviewers. Editor cannot truly know – either scientifically or for the quality of their ethics – all the reviewers he has in his pool, and this can lead to a number of biases.

Authors are often encouraged or even requested to propose possible reviewers for their papers. For a good journal, this can be a way to accurately identify the sub-domain of the manuscript and will make it easier to find recognized experts. It is clearly good when the original reviewer declines to submit a report because he or she isn't close enough to the field of research, but can still identify one or more experts that the editor wouldn't have been able to identify. With lower-quality journals, the editor may decide to lazily follow the suggestions of the authors, at the risk of being oriented to friends or to people from a same small community trying to exaggerate the importance of their own field. Even worse, the suggested reviewers can sometimes be the authors themselves, hidden behind an electronic alias bearing the name of a supposed specialist.

The anonymous refereeing process can have other drawbacks as well. More than a few researchers know someone whose paper was rejected, but whose idea miraculously reappears under the name of a colleague who was suggested as a reviewer. The problem has been reduced by the development of sites that publish preprints. The practice of "anonymous" reviewers asking that a reference be added to their own works is relatively common, but often transparent.

Recognized journals, predatory ones, and other bad practices

The pressure to publish – the famous "publish or perish" – and role of chance in any expertise means that any reasonable manuscript free of gross errors will end up being published, even after rejection by one or more journals. Rather than being abnormal, this explains how a hierarchy of journals can be established. Because on-line publications are truly cheap compared printed ones, "predatory" journals now appear claiming to be "peer-reviewed". Such publications, easily recognized by true scientists, publish for a fee any work claiming to be for a scientific audience, and provide a vague "referee report".

These deceptive practices crop up because there is a considerable growth of higher education at a world level. In most cases, the costs for publishing in such predatory reviews will be billed to the university itself, and for faculty from peripheral institutions, publishing can be profitable in terms of recognition and career advancement. Similar practices in the refereeing process also occasionally occur in the humanities and social sciences, where nonsense texts that resemble academic work, sometimes computer-generated, can pass the "refereeing process".

Some paths for improvement

"Peer-reviewing" is essential for the advancement of science. However, the standard of publications with reviewers is affected by the development of on-line publications and the growing number of papers. Some paths are worth being considered to improve the system:

• Reviewer reports can be made available on-line, helping readers understand the context of the work and of its publication, as any critics of a work, in art and in science as well. Such a practice is now being tested by some journals. It is also a way to establish the quality of a journal through its ability to select appropriate and sharp referees.
• Some high-level reviews have considered a temporary electronic deposit, open to comments by researchers in the appropriate field, before deciding on the validation as a publication. This would be available only for voluntary readers, and when the comments are favourable. Most likely, such a system would work only for top papers that are likely to attract known specialists as readers.
• Requiring that submissions and the comments they receive be tracked from journal to journal would limit the publication of lower-quality work. Currently, a manuscript that's rejected by one review is sent free of critiques when submitted to another, except when the two are within the same editorial group. Showing that one improved a piece of research in response to criticism isn't an admission of failure, but natural and laudable.

As an author, I would be more confident in receiving a fair evaluation from a journal for a manuscript if I could show how it was improved by the comments received during a prior submission. Presently, I have the feeling that I would be infringing the intellectual property of the first journal and its referees, and so refrain from attaching such material.