Special education professor advocates for steps to combat replication crisis in research

Replication of scientific findings has been a cornerstone of validating research for generations, yet it happens so infrequently that many have claimed science is in a replication crisis. A University of Kansas special education professor has co-authored a study on replication, its effects on the field and students, and suggests a more dynamic approach to research could help address the paucity of replication.

Jason Travers, assistant professor of special education, was lead author of an article examining how the lack of research replications can negatively affect special education, and he argues that single-case experimental research design can complement group experiments to address the shortage. The study is the lead article in a special issue of the journal Remedial and Special Education—edited by professors Kathleen Lane and Karrie Shogren of KU—dedicated to the replication crisis.

Like nearly all academic fields, special education researchers face several challenges that make replicating research findings a challenging prospect. The article points out several challenges such as publishers' bias of novel findings, unrealistic expectations for researchers to publish a high quantity of original studies, a bias among the public and researchers for "striking" findings, lack of research funding for replication studies and many others. Travers co-authored the article with Bryan Cook of the University of Hawaii, William Therrien of the University of Virginia and Michael Coyne of the University of Connecticut.

While those problems are not unique to special education, they are troublesome because the field regularly develops interventions that educators will use for children with all manner of disabilities throughout the nation, Travers said. Without rigorous verification of previous findings, children could be subjected to interventions that are not truly effective or possibly even harmful.

For many years the standard approach to research testing and developing educational interventions has been group experimental designs. Special education is no different, but it is unique in a different respect.

"In special education we have a good deal of people who do research that utilizes the typical group paradigm. That's normal experimental work and a good chunk of what special education research looks like," Travers said. "The problem is that it can be very difficult to get a large group of participants for low-incidence disabilities, and group designs don't tell us much about who did and did not respond well to an intervention."

For example, autism, which is widely publicized and studied, is still technically a low-incidence disability. It can be very difficult to find 100 or more youths with autism to take part in a randomized group study. And even when possible, the range of variance among the youths—some would be very highly functioning and fully communicative while others would not, not to mention age, cultural, socioeconomic and other differences—make it very difficult to study the effectiveness of an intervention with a certain goal, such as improving communication.

Travers and co-authors therefore advocate for more use of a single-case research design, which could both lead to effective interventions and make replication less cumbersome.

A single-case experiment usually includes with one to five learners who are systematically introduced to an intervention and observed for changes in responding. The intervention may be removed and re-introduced to observe effects, such as the frequency of a response, to determine whether the intervention alters that response. The method can be used with a single learner at a time and many times over with individuals with varying levels of disability. The result is increased clarity about what types of individuals respond well or poorly to a certain intervention, and what variables increase or diminish responses for individuals as opposed to drawing conclusions from average responses by a group of individuals.

"In group experiments, causal effects are inferred. In single-case design, researchers explicitly demonstrate a causal relationship. They manipulate an independent variable to observe effects on dependent variables in ways that clarifies who does and doesn't respond to an intervention. We think this has some implications for the replication crisis," Travers said.

Increasing the number of single-case experiments could help increase the feasibility of replication by making it faster, more cost-effective and more practical to attempt. For example, special education research participants are most commonly students in public schools. In order to enroll them in a study, permission is needed from a university, school district, principals, teachers, parents and sometimes students themselves.

"Substantial money, time and other resources are often necessary to conduct a large group experiment of an intervention in applied settings," Travers said. "That alone affects how many studies can be conducted and therefore how many findings can be replicated before we can be confident we have a valid answer to a question."

The efficacy of single-case research design has long been established, yet it is not widely used. Medical and developmental disabilities researchers regularly use the method, but it is not as common in other areas of special education such as reading disorders and other disabilities, nor in other fields of scientific research. The method is sometimes confused with a qualitative case study, but Travers said single-case experiments differ in that variables are manipulated and effects are observed, as opposed to describing in detail what happens in a research setting.

Travers and colleagues argue that single-case experiments can work in a complementary way with group design research. Group design studies make valuable findings, but can be extremely difficult to replicate as they often take a skilled research team several years and millions of dollars to complete. By implementing a single-case design to replicate findings, researchers could not only do so more efficiently, they could add to the understanding of findings, even if they don't necessarily confirm the findings directly. Further, a collection of single-case experiments could inform researchers about a population from which samples could be drawn for large group studies. Conversely, results from large group studies can inform the selection of participants for single-case experiments to clarify what seems to influence responding.

Those advantages could also lend themselves to replicating research throughout the sciences, which could simultaneously help address the widespread lack of replication and bolster the validity of all manner of academic findings.