Study assesses the strengths of research synthesis over 40 years

Figuring out what is true in science when researchers are bombarded with information from many different studies is a challenge. A new paper, published in Nature, reveals that the power of meta-analysis in research synthesis over the past 40 years has transformed scientific thinking and research approaches. Meta-analysis has also become invaluable to making advances in many scientific fields, including medicine and ecology.

Meta-analysis is the quantitative, scientific synthesis of research results from different studies investigating the same question. Since the approach (and the term) were first introduced in the 1970s, meta-analysis has been used to resolve seemingly contradictory research outcomes, to identify where more research is needed, and to tell scientists when no more studies are needed because the answers have become clear.

"Meta-analysis is the grandmother of Big Data and Open Science. Our paper illustrates how meta-analysis is used in different scientific fields, why it has become so important, and what criticisms and limitations it faces," said co-author Jessica Gurevitch, Ph.D., Professor of Ecology & Evolution as Stony Brook University. "It is a statistical and scientific approach to resolving questions and reaching generalizations, but it is not magic, and can't produce data on its own where none exists."

In the paper "Meta-analysis and the science of research synthesis," Professor Gurevitch and colleagues from the University of New South Wales in Australia and Newcastle University and Royal Holloway in the United Kingdom review research over the 40 years to illustrate the accomplishments and challenges of the method, along with new advances and direction of meta-analysis in 21st Century scientific research.

Even when trying to discover things that have been the subject of many experiments, such as whether boys or girls are better at math or the best treatment immediately after a stroke, Professor Gurevitch explains, the use of meta-analysis is essential to reduce biases and strengthen methods to determine the answer, or answers.

"By bringing the scientific method into the synthesis of results across independent studies, we can ask—and answer—questions we could never hope to resolve before, while new methods in meta-analysis open doors to the resolution of seemingly intractable problems," adds Shinichi Nakagawa, a co-author from the University of New South Wales.

Citing some of the examples of what recent advances in meta-analysis have taught us, Professor Gurevitch explains that by combining experiments in a rigorous manner, "In medicine, we have been able to compare the effectiveness of treatments that have never been directly compared in any one study, and we can save lives by understanding what works long before it is apparent in any single experiment.

"In ecology and evolutionary biology, we can evaluate patterns across wide expanses of space and a diversity of animals and plants that are more than any one researcher could study in several lifetimes. In conservation, we may find out what measures really work best to achieve the goals of preserving biodiversity and threatened ecosystems. "