New tool tackles reproducibility crisis in science

February 13, 2019 by Emily Ayshford, University of Chicago
Researchers with the Institute for Molecular Engineering and Argonne National Laboratory have developed a platform to provide searchable data of scientists' publications in order to address the issue of reproducibility. Credit:

The scientific community has been struggling with the problem of data reproducibility—a key step in the process that guides how most scientists create knowledge in their field.

According to a recent survey, more than 70 percent of researchers have tried and failed to reproduce another scientist's published experiments. More than half even failed to reproduce their own investigations successfully. That begs the question: If an experiment or a simulation cannot be reproduced, was it successful in the first place?

Researchers with the Institute for Molecular Engineering at the University of Chicago and Argonne National Laboratory aim to help solve this problem with a new software platform that allows scientists to share the of each of their publications in a searchable way. Over the last two years they developed a tool for curating, discovering and exploring reproducible scientific papers called Qresp, which is now available for public use.

"Our goal is to speed up the and reduce the time needed to share knowledge among researchers," said Giulia Galli, the Liew Family Professor of Molecular Engineering. "By making data available and searchable, we are hoping to make it easier for researchers to reproduce results."

The process of reproducing scientific results remains a complex issue. Published papers, which are available online as PDFs, often don't include enough information about the resulting data and processes for others to reproduce the results, and data are often not made available to the .

"Many papers do not include sufficient details to be able to reproduce the data," said Marco Govoni, assistant scientist at Argonne and a visiting scientist at the University of Chicago. "And oftentimes the majority of data obtained and used in the paper are not available at all. To get that data, sometimes you need to write to the authors of the paper. It should not be this difficult, and it should not be an ad hoc process."

With Qresp, researchers hope to help relieve some of the current difficulties in making data open and reproducible. The software guides users through the process of organizing and sharing their data, including datasets and charts. All fields are customizable, letting researchers curate their data in the best way according to the paper they have written. The platform is also available for anyone who wants to explore data shared by other researchers.

Though there have been several efforts to manage large sets of data, most of those efforts rely on a central repository, while Qresp relies on a distributed model. Within the platform, researchers do not upload data but rather host their own curated data and decide what they want to share. That way, Qresp provides a scalable solution to sharing data, Govoni said.

Researchers who might not want to take this extra step to share their data right after publication should consider the benefits of organizing and sharing data for their own group, Galli said. Students in her group now automatically curate their data in Qresp as one more step in their scientific research. Qresp facilitates transfer of information and knowledge between projects carried out by different students, and between researchers who stay and those who leave the group.

"The whole data sharing in the group has become much more efficient," said Galli, who is also a professor of chemistry and a senior scientist at Argonne.

With the use and adoption of Qresp by a broad community of researchers, published papers may become much more interactive—a living interface where, by clicking on an image, a researcher can see the dataset behind the results.

"We want to raise the bar for reproducing scientific results, and we want to move beyond publishing PDFs and into sharing research that is much more interactive and useful," Govoni said.

Explore further: Data sharing can offer help in science's reproducibility crisis

More information: Marco Govoni et al. Qresp, a tool for curating, discovering and exploring reproducible scientific papers, Scientific Data (2019). DOI: 10.1038/sdata.2019.2

Related Stories

Ecologists ask: Should we be more transparent with data?

October 26, 2018

Computational reproducibility—the ability to accurately reproduce outcomes from data sets using the same code and software—will be an increasingly important factor in future scientific studies according to a new paper ...

Recommended for you

Archaeologists discover Incan tomb in Peru

February 16, 2019

Peruvian archaeologists discovered an Incan tomb in the north of the country where an elite member of the pre-Columbian empire was buried, one of the investigators announced Friday.

Where is the universe hiding its missing mass?

February 15, 2019

Astronomers have spent decades looking for something that sounds like it would be hard to miss: about a third of the "normal" matter in the Universe. New results from NASA's Chandra X-ray Observatory may have helped them ...

What rising seas mean for local economies

February 15, 2019

Impacts from climate change are not always easy to see. But for many local businesses in coastal communities across the United States, the evidence is right outside their doors—or in their parking lots.

The friendly extortioner takes it all

February 15, 2019

Cooperating with other people makes many things easier. However, competition is also a characteristic aspect of our society. In their struggle for contracts and positions, people have to be more successful than their competitors ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.