Dealing with the reproducibility crisis: what can ECRs do about it?
Unless you've been living under a rock (no judgment, by the way), I'm sure you've heard about the reproducibility crisis in scientific research. In 2016, two posts on this blog covered what the main causes of irreproducibility are and what can be done, and how we can reform scientific publishing to value integrity. To briefly recap, a study published in PLOS Biology noted that half of preclinical research is not reproducible. The estimated price tag on this irreproducibility is alarming—a whopping $28 billion. In my opinion, however, the most troubling cost of this crisis is its impact on public trust in science.
In January, the editors of this blog wrote an inspiring post outlining the issue of public misinformation and mistrust in science, and what we as ECRs can do to ameliorate the situation. The editors called on ECRs to contribute by communicating good science. While this is an excellent way to help foster public interest and involvement in science, I think we can do even more by actively tackling the reproducibility crisis. The potential benefits include boosting public trust in scientific research, as well as saving resources and promoting a more collaborative landscape. We might even get better science out of it. Now that's really hitting several birds with one stone!
OK, so what can I do about reproducibility?
Assuming I've convinced you to be concerned about reproducibility, you might now be wondering what you can do about it as an ECR. There's a whole lot you can do as a researcher to improve reproducibility in your own lab, including setting up reproducibility measures, like having a lab mate independently repeat an experiment, standardizing and sharing your protocols and methods, or pre-registering your study. Other suggestions for improving reproducibility include better mentoring and teaching, improved understanding of statistics, and more robust experimental design. While these are all great ideas, I believe a large part of the responsibility for combatting reproducibility rests with publishers and institutions. As the gate-keepers of academic research, these organizations have great potential for creating impactful change in the process of science.
Publishers' efforts in open science and reproducibility
Luckily, some publishers seem to agree. In January, PLOS Biology released a statement outlining their new policy on 'complementary research'. If your manuscript is an extension or confirmation of a recently published study (AKA being 'scooped'), it will be considered for publication. Encouraging publication of replication studies is important aspect of improving reproducibility. In March of last year, Nature announced a 'transparency upgrade' for their journals, with details on their now 5-year-old reproducibility checklist for authors and editors, as well as other efforts such as abolishing length limits in methods sections, protocol sharing via an open repository, and a commitment to TOP (Transparency and Openness Promotion) guidelines. Other publishers such as eLife are tackling reproducibility by promoting open science and encouraging responsible scientific behaviour.
Teamwork makes the dream work
Openness and collaboration are at the heart of reproducibility—the more transparent and clear we are about the research we have done, the more likely it is another lab will be able to reproduce it. That being said, even when three different labs spent years coordinating their research on ageing related compounds in worms, they were only able to reduce inter-lab variability but not run-to-run variability within individual labs. As a result of their efforts to eliminate all other sources of variability, however, they found an interesting biological phenomenon which all three labs are now following up on as a team. Instead of competing and working against each other when the outcome of their experiments were contradictory, these scientists had the opportunity to work together in an open, collaborative environment, allowing them to focus on the biology instead of arguing over who's right.
How institutions can help
But what is the incentive to collaborate? While the publish-or-perish culture of academia is alive and well, there is little incentive for openness and collaboration—not just between lab groups, but also between individuals in the same group. This hyper-competition is fueled by the acute awareness shared by many junior academics of the small number of senior-level academic positions available. Institutions can play a key role in lowering this competition by better aligning the number of available senior academic roles to the number of Ph.D.s accepted each year. Additionally, we could develop different Ph.D. curricula (such as the R3 Program at Johns Hopkins) with a focus on interdisciplinary learning and critical thinking, which would prepare graduates for roles outside of academia. Making Ph.D. students aware of all the options they have outside of academia could help lower the number of post-docs struggling to find a permanent university position. Most importantly, however, institutions should focus on rewarding good science. Instead of awarding promotions and tenured positions based on high-impact publications, institutions could focus on a researcher's dedication to open science, their commitment to reproducibility, the quality of their work, and article-level metrics, to name a few.
As ECRs we have the potential to create change and improve the process and practice of science. We can start by doing better science ourselves and shifting our own lab practices to increase reproducibility. We can go further by participating in events that aim to create innovations promoting open science and collaboration, such as the eLife Innovation Sprint and the MIT Better Science Ideathon. Our ideas and efforts will shape the future of scientific research. If we all work towards producing better, reproducible research, then increased public trust in science will be just one of the many benefits we'll see along the way.
This story is republished courtesy of PLOS Blogs: blogs.plos.org.