Trials testing new educational methods in schools 'often fail to produce useful evidence'

March 15, 2019, University of York

Educational trials aimed at boosting academic achievement in schools are often uninformative, new research suggests.

The new study, published in the journal Educational Researcher, found that 40% of large-scale randomised controlled trials (RCTs) in the UK and the US failed to produce any evidence as to whether an educational helped to boost academic attainment or not.

The researchers evaluated 141 trials involving more than one million students, which tested schemes ranging from whether providing free school breakfasts raises grades in Maths and English, to whether playing chess improves numeracy skills.

The trials, which were carried out by the charitable organisation the Education Endowment Foundation (EEF) in the UK and the National Center for Education Evaluation and Regional Assistance (NCEE) in the US, are expensive—with costs often exceeding £500,000.

The authors of the study argue that more research is urgently needed to understand why RCTs in education are so often uninformative.

Lead author of the research, Dr. Hugues Lortie-Forgues, from the Department of Education at the University of York, UK, said: "Just like in medicine, trials of educational interventions are an important way to allow policy makers and teachers to make informed decisions about how to improve education. However, many of these trials are currently not fulfilling their main aim of demonstrating which interventions are effective and which are not."

"Further research to investigate the reasons for this should be a priority. These organisations are trying to achieve something positive and reform is urgently needed to help them to do so."

In recent years there have been a growing number of RCTs conducted in . For example, in the UK, the Education Endowment Foundation (EEF) has commissioned more than 191 trials since 2012.

The researchers cite possible reasons why current trials may be ineffective, including:

  • The interventions being tested may not be suitable for trial in the first place.
  • Interventions may not be being correctly implemented during trials—for example due to inadequate training of teachers in the methods being tested.
  • The trials themselves may be poorly designed

The authors suggest a series of changes that could make the more informative, including higher-standards when considering which new initiatives are trialled.

Rigorous Large-Scale Educational RCTs are Often Uninformative: Should We Be Concerned? Is published in the journal Educational Researcher. The study was carried out in collaboration with researchers at Loughborough University.

Explore further: Publication characteristics tied to treatment effects

More information: Hugues Lortie-Forgues et al, Rigorous Large-Scale Educational RCTs Are Often Uninformative: Should We Be Concerned?, Educational Researcher (2019). DOI: 10.3102/0013189X19832850

Related Stories

Publication characteristics tied to treatment effects

September 6, 2018

(HealthDay)—Published trials have larger treatment effects than unpublished trials, while trials published in a language other than English have larger treatment effects versus those published in English, according to research ...

Spine education seems ineffective in pain prevention

December 10, 2012

(HealthDay)—Educational interventions, mainly focused on a biomechanical/biomedical model, do not seem to be effective in preventing low back pain, according to a review published in the December issue of the European Spine ...

Why we should (carefully) consider paying kids to learn

February 14, 2019

Over the past 15 years, we've seen a decline in the performance of Australian school students on international tests. On the Programme for International Student Assessment (PISA), Australia ranks a disappointing 20th in mathematics ...

Discontinuation of randomized clinical trials common

March 11, 2014

Approximately 25 percent of about 1,000 randomized clinical trials initiated between 2000 and 2003 were discontinued, with the most common reason cited being poor recruitment of volunteers; and less than half of these trials ...

Effect sizes uncertain for meds versus placebo in knee OA

December 29, 2018

(HealthDay)—For patients with knee osteoarthritis and at least 12 months of follow-up, there is uncertainty around estimates of effect size for the change in pain in association with medications versus placebo, according ...

Recommended for you

Study reveals properties of a Type Ib supernova in NGC 4080

March 25, 2019

A recent study conducted by astronomers has revealed important observational properties of a Type Ib supernova designated MASTER OT J120451.50+265946.6, which exploded in the galaxy NGC 4080. The research, presented in a ...

Catalyst advance removes pollutants at low temperatures

March 25, 2019

Researchers at Washington State University, University of New Mexico, Eindhoven University of Technology, and Pacific Northwest National Laboratory have developed a catalyst that can both withstand high temperatures and convert ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.