Study completes new analysis of patents to refute earlier claim that research has lost its innovative drive

A high-profile study made headlines in 2023 stating that the scientific and innovation system is producing less and less completely new knowledge. Researchers at the University of Basel are now refuting this claim, at least for patents: It is based on a measurement error.

The discovery of mRNA in the 1960s was groundbreaking. Suddenly there were completely new findings that ushered in new developments. This kind of discovery is described as "disruptive." In contrast, research findings are "consolidating" when they build upon existing knowledge. They are also important, as the example of the mRNA vaccines that helped to overcome the COVID-19 pandemic shows—but they would not have existed without the prior groundwork on mRNA.

Both types of discovery are therefore needed—disruptive and consolidating. However, a study published in the journal Nature in 2023 claims that this relationship is now out of balance, meaning that the scientific innovation system is producing fewer and fewer groundbreaking discoveries.

The stream of citations

For the Nature study, US researchers analyzed millions of scientific publications from the years 1945–2010 and patents from the years 1976–2010 using the CD index. This index assigns values between 1 (totally disruptive) and -1 (totally consolidating). The rating is based on the extent to which a given scientific work or patent is cited together with other prior works.

For a patent this means the following: if subsequent patents only cite this patent—let's call it patent C—but not the patents prior to patent C, patent C is classified as disruptive and represents the start of a stream of citations, so to speak.

In contrast, the patent C in question is part of the "consolidating" category if subsequent patents (D, E, F) also cite prior patents (A, B), meaning that patent C is not entirely novel.

Artificially disruptive

By concluding that disruptive research has sharply declined, the Nature study called the innovative capacity of the entire scientific system into question. Two researchers from the University of Basel, Dr. Christian Rutzer from the Center for International Economics and Business (CIEB) and economics Professor Rolf Weder, were skeptical from the start—and when they checked the calculations, they found serious measurement errors.

Together with Professor Jeffrey Macher (Georgetown University), who was a visiting professor at the Faculty of Business and Economics in spring 2023, they began their own analysis for patents. Their results were recently published in the journal Research Policy.

The measurement error in the Nature study stems from its exclusion of citations to patents published prior to 1976. "This time constraint strongly impacts the results," states Dr. Rutzer. "Most patents from the early 1980s cite patents which were published before 1976. If you cut out these citations, many of these patents become disruptive—not because they are, but because many citations to prior patents are not taken into account."

Later, in the 1990s, patents cite patents from before 1976 less and less frequently, meaning that the number of patents incorrectly classified as disruptive also dropped. From 2005, the measurement error tends towards zero.

Correcting the error

By also taking citations to patents published before 1976 into account, Macher, Rutzer and Weder demonstrate that the time constraint significantly distorts the results. The values quickly changed: the average patent disruptiveness in 1980 was no longer 0.39 as in the Nature study, but much lower at 0.09—and it fell only slightly to 0.04 in 2005. The authors also demonstrate that the number of strongly disruptive patents has even increased over the long term.

Co-author Rolf Weder puts the correction of the Nature study into perspective: "Scientific papers can contain errors or one-sided interpretations, but it is important that justified critique is published swiftly—as in our case." This demonstrates that scientific self-regulation is working.