Pediatric clinical studies appear prone to bias, Hopkins review shows

Jul 12, 2010

A Johns Hopkins review of nearly 150 randomized controlled trials on children — all published in well-regarded medical journals — reveals that 40 to 60 percent of the studies either failed to take steps to minimize risk for bias or to at least properly describe those measures.

A report of the team's findings in the August issue of Pediatrics shows that experimental trials sponsored by pharmaceutical or medical-device makers, along with studies that are not registered in a public-access database, had higher risk for bias. So were trials that evaluate the effects of behavioral therapies rather than medication, the report states.

"There are thousands of pediatric trials going on in the world right now and given the risk that comes from distorted findings, we must ensure vigilance in how these studies are designed, conducted and judged," says lead investigator Michael Crocetti, M.D., M.P.H., a at Johns Hopkins Children's Center. "Our review is intended as a step in that direction."

Considered the gold standard of medical research, the hallmark of double-blind (RTC) is a design that rules out or accounts for actual or potential bias. Results of such studies, when peer-reviewed and published in reputable medical journals, can influence the practice of medicine and . A poorly designed or executed trial can therefore lead researchers to erroneous conclusions about the effectiveness of a drug or a procedure.

Citing the degree of bias risk in the studies they reviewed, the researchers caution pediatricians to be critical readers of studies, even in highly respected journals.

The investigators advise that when reading a report on a trial, pediatricians should not merely look at the bottom line but ask two essential questions: How did the researchers reach the conclusion? and Was their analysis unbiased?

Doctors should apply "smell tests," common sense and skeptical judgment about whether the conclusions fit the data, especially when a study boasts dramatic effects or drastic improvement.

Crocetti and colleagues used the Cochrane Collaboration tool, which assesses risk for bias along six critical aspects including randomization — randomly assigning patients to different treatments — and masking, the degree to which neither the patient nor the doctor knows which group of patients is receiving an active drug or intervention versus a placebo.

Investigators say that by analyzing each clinical trial along these and four other dimensions, the Cochrane Collaboration can answer what are perhaps the most important questions in medical research: How strong is the causal relationship between the therapy and the effect? and How valid are the conclusions made about the effect of the therapy?

Overall, 41 percent of the 146 trials in the review had improper or poorly described randomization techniques. Industry-funded trials were six times more likely to have high risk for biased randomization than government-funded trials or those funded by nonprofit organizations. And past research, the investigators point out, has shown that industry-funded trials are four to five times more likely to recommend an experimental drug.

"Industry funding is an important driver of medical discovery, but it is critical for investigators involved in such trials to ensure not only that the studies are conceived and executed cautiously with minimum risk for bias, but that any precautions taken against bias are also reported transparently," Crocetti says.

Trial registration — and the transparency commitment it reflects — is a key step in reducing bias or its influence, the researchers say. In their evaluation, registered trials were nearly 70 percent more likely to have robust randomization than non-registered trials, probably because the registration process itself forces researchers to answer many questions related to trial design and execution. Therefore, the investigators say, registering pediatric trials in the public domain will not only increase transparency but, in the long run, improve the validity of their results. In 2005, the International Committee of Medical Journal Editors called for registering all medical trials involving human subjects with www.ClinicalTrials.gov, a massive public repository of more than 70,000 trials from all over the world. However, less than 60 percent of the pediatric trials in the review were registered there.

The researchers also found that most of the trials (57 percent) either failed to use proper techniques that ensure anonymity or "blinding" to the type of treatment a patient gets, or they failed to clearly describe these techniques. The technique, called allocation concealment, ensures that neither the researcher nor the patient can guess which treatment they will get. The method also helps ensure that the treatment of one subject will not reveal to either scientists or the patients clues about the treatment of the next subject. Trials involving behavioral therapies were four times more likely to have this problem.

Overall, nearly 20 percent of the trials used improper masking techniques to ensure that neither the patient nor the researchers know which treatment went to which patient.

The 146 trials in the review appeared between 2007 and 2008 in the five leading pediatric journals — Pediatrics, Pediatric Infectious Diseases, Journal of Pediatrics, Archives of Pediatrics and Adolescent Medicine and the Journal of the American Academy of Child and Adolescent Psychiatry — and the three highest-ranked general medical journals — JAMA, The New England Journal of Medicine and The Lancet.

Explore further: Cold cash just keeps washing in from ALS challenge

More information: www.pediatrics.org

Provided by Johns Hopkins Medical Institutions

5 /5 (1 vote)
add to favorites email to friend print save as pdf

Related Stories

Extensive publication bias for Phase I drug trials

Feb 18, 2009

A study published in this week's issue of the open-access journal PLoS Medicine suggests that, in comparison to other types of trials, the results of Phase I drug trials are far less likely to be published.

Half of trials supporting FDA applications go unpublished

Sep 23, 2008

Over half of all supporting trials for FDA-approved drugs remained unpublished 5 years after approval, says new research published in this week's PLoS Medicine. The most important trials determining efficacy, and those with s ...

Doubts cast on credibility of some published clinical trials

Jul 02, 2009

Randomised Controlled Trials (RCTs) are considered the 'gold standard' research method for assessing new medical treatments. But research published in BioMed Central's open access journal Trials shows that the design of a r ...

Recommended for you

Cold cash just keeps washing in from ALS challenge

3 hours ago

In the couple of hours it took an official from the ALS Association to return a reporter's call for comment, the group's ubiquitous "ice bucket challenge" had brought in a few million more dollars.

Medtronic spends $350M on another European deal

19 hours ago

U.S. medical device maker Medtronic is building stronger ties to Europe, a couple months after announcing a $42.9 billion acquisition that involves moving its main executive offices across the Atlantic, where it can get a ...

Mind over matter for people with disabilities

Aug 26, 2014

People with serious physical disabilities are unable to do the everyday things that most of us take for granted despite having the will – and the brainpower – to do so. This is changing thanks to European ...

Ukraine's former world's tallest man dies

Aug 25, 2014

Ukraine's tallest man, who briefly held the world record but gave it up to live as a recluse, has died due to complications from the condition that saw him never stop growing, local media reported Monday.

User comments : 0