'Quantitative literacy' would prevent unsound research policy

Credit: CC0 Public Domain

Research impact is measured in different ways. However, these indicators are often based on dubious calculations, says Ludo Waltman.

Policymakers try in various ways to evaluate how well researchers and research institutions are performing. How often is a researcher cited by colleagues in the field? How great is his or her impact, as measured in the h-index? And how well is a university performing in comparison with other institutions at home and abroad?

However, these indicators and rankings are often based on dubious calculations, says Ludo Waltman in his inaugural lecture. This Professor of Quantitative Science Studies argues that university rankings combine incompatible indicators. "These rankings measure in part the size of universities and in part their relative performances in relation to their size. These perspectives are then thrown into one pile. Then no one really understands what exactly the rankings are telling us." Other rankings also juggle the figures.

According to Waltman, the danger is that simplistic indicators will have a on research policy, for instance if researchers or are assessed on the basis of these indicators. He believes that a higher level of quantitative is needed to bring about change. A better understanding of quantitative analyses—as well as of the conclusions that are reached from these analyses—would make a world of difference.

Waltman believes that quantitative research places too much value on , "Which means that research results are presented in an unnatural, binary way, so as significant or not." Furthermore, in these calculations, correlation is often confused with causality. Much could be gained from training that increases levels of quantitative literacy, says Waltman. "The use of data-driven methods in the specific context of policy development doesn't appear to receive the systematic attention that it deserves."

"Wherever, possible, research policy should make use of the findings of quantitative science," says Waltman in his inaugural lecture. "But it is unrealistic to think that we can provide clear, quantitative answers to the many questions that exist. I think that we should move away from the idea that we can answer these kinds of complex question with a simple yes or no."

Explore further

College rankings go under the microscope

Provided by Leiden University
Citation: 'Quantitative literacy' would prevent unsound research policy (2019, June 24) retrieved 23 September 2021 from https://phys.org/news/2019-06-quantitative-literacy-unsound-policy.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Feedback to editors