International survey finds researchers want research outputs to be accompanied by better quality indicators

Ten years on from the influential peer review survey of 2009, new findings show researchers have confidence in peer review, but strains on the system—including the increasing volume of research outputs and platforms to access research—are causing them to worry about quality and reliability.

The survey found that researchers don't want to replace the peer review process, as 75 percent are satisfied with it (compared to 69 percent in 2009). Researchers want to improve and extend the system. It is a question of when peer review should be used and knowing when it has been used. Over a third—37 percent—admitted they had doubts over the quality of at least some research outputs they had encountered in the last week.

The study comes from the same teams from Sense about Science and Elsevier, as 2009, with over 3,000 respondents, randomly selected from the Scopus database of over 3 million published researchers.

Commenting on the survey, Sense about Science Director Tracey Brown said, "What this survey shows is that our inroads into getting wider understanding of peer review are now not enough for research publishing in the 2020s, or for people to understand what quality checks have been done.

"We urgently need to see a common language and greater transparency about what has been reviewed. Does it include data? What criteria were used? Is it even a published study? Remember many research users are arriving at information via search engines, not academic portals."

Other results include:

• Respondents want more information available alongside research papers, including an indicator to show whether someone else had tried to reproduce the research (82 percent). (see fig. 10, p20 of full report).
• Most researchers (76 percent) believe that data and supporting material that accompany research articles should be reviewed. (see fig. 9, p19 of full report).
• Two-thirds of participants feel that reviewers would benefit from clearer guidance on reviewing criteria, while many felt that recognition of reviewing work was important for maintenance of a healthy peer review system, most commonly from employers (see fig. 12, p22 of full report).
• Few researchers (38 percent) agree that the public understand the concept of peer review. There is strong support (77 percent) for a clear indication that material has been reviewed; and most (70 percent) feel explaining research context, and the implications of research in lay terms would be most helpful.
• Despite the trend towards using technology to evaluate evidence, few respondents (8 percent) felt the evaluation that relied on artificial intelligence would qualify as peer review.

There are four areas for urgent action:

1. Amid rising volume of research outputs and information platforms, researchers want improved signals, be able to cross-check work easily.
2. Better training, information, and more career-based recognition is raised by researchers in this and other studies. Finding a way to remove the current inconsistencies in reviewer instructions is paramount.
3. Agreement about the role of technology in peer review, e.g., using it to manage the rising volume of submissions and alleviate the burden on reviewers, without losing the benefits of human judgement? Without discussion, the use of AI may further disrupt people's ability to trust content.
4. Ensure that research is put in context: articles include easy-to understand explanations of research; and a common language is used to communicate to both researchers and public alike what has been reviewed and the type of review done.

Adrian Mulligan, Research Director for Customer Insights, Elsevier, said, "This study highlights the growing pressure being placed on research communication and importantly the value of peer review. Maintaining the integrity of the peer review system is paramount so it's important various stakeholders work together to ensure reviewers receive recognition, have clear guidance and quality tools to support them in their roles."