Scientific reproducibility is hampered by a lack of specificity of the material resources

September 5, 2013

A key requirement when performing scientific experiments is the accessibility of material resources, including the reagents or model organisms, needed to address a specific hypothesis. The published scientific literature is a source of this valuable information, but frequently lacks sufficient detail to the extent that researchers are unable to identify material resources used to perform experiments.

A study, published today in PeerJ, demonstrates the magnitude of the problem – a problem that negatively affects the ability of scientists to reproduce and extend reported studies. The study showed that a large number of scientific resources are unidentifiable based on the information reported within the .

"The stories we tell in are not necessarily instructions for ." said Melissa Haendel, Ph.D., an ontologist and assistant professor in the Library and Department of Medical Informatics & Clinical Epidemiology at Oregon Health & Science University and senior author on the study. "This study illuminates how if we aim to use the literature as the scientific basis for reproducibility, then we have to get a lot more specific."

The study, led by Haendel and Nicole Vasilevsky, Ph.D., project manager and biocurator in Oregon Health & Science University's Ontology Development Group, examined nearly 240 articles from more than 80 journals spanning five disciplines: neuroscience, immunology, cell biology, developmental biology and general science. The articles were evaluated to determine if the reported research resources could be uniquely identified based on the information that was provided in each article, its supplemental data, or prior references. Specific criteria were developed to determine if antibodies, cell lines, constructs, model organisms, and knockdown were identifiable. Based on these criteria, Haendel, Vasilevsky and their team of researchers also developed guidelines for reporting of research resources. These guidelines are available online (http://www.force11.org/node/4433) and can be used as a new data standard by authors, reviewers, publishers, and other data contributors to aid reproducibility.

The study showed that just under 50 percent of scientific resources used in previously published articles were unidentifiable, a percentage which varied across resource types and disciplines. The study also found no increased level of identification in journals that had more stringent reporting guidelines.

"We hope that quantifying the problem through this study will highlight to the research community that there is a significant and pressing need to make material resource information more accessible going forward," said Vasilevsky.

Explore further: Pressure for positive results puts science under threat, study shows

More information: Vasilevsky et al. (2013), On the reproducibility of science: unique identification of research resources in the biomedical literature. PeerJ 1:e148; DOI: 10.7717/peerj.148

Related Stories

Recommended for you

Earliest evidence of reproduction in a complex organism

August 3, 2015

Researchers led by the University of Cambridge have found the earliest example of reproduction in a complex organism. Their new study has found that some organisms known as rangeomorphs, which lived 565 million years ago, ...

Model shows how surge in wealth inequality may be reversed

July 30, 2015

(Phys.org)—For many Americans, the single biggest problem facing the country is the growing wealth inequality. Based on income tax data, wealth inequality in the US has steadily increased since the mid-1980s, with the top ...

French teen finds 560,000 year-old tooth (Update)

July 28, 2015

A 16-year-old French volunteer archaeologist has found an adult tooth dating back around 560,000 years in southwestern France, in what researchers hailed as a "major discovery" Tuesday.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.