Pediatric clinical studies appear prone to bias, Hopkins review shows

Jul 12, 2010

A Johns Hopkins review of nearly 150 randomized controlled trials on children — all published in well-regarded medical journals — reveals that 40 to 60 percent of the studies either failed to take steps to minimize risk for bias or to at least properly describe those measures.

A report of the team's findings in the August issue of Pediatrics shows that experimental trials sponsored by pharmaceutical or medical-device makers, along with studies that are not registered in a public-access database, had higher risk for bias. So were trials that evaluate the effects of behavioral therapies rather than medication, the report states.

"There are thousands of pediatric trials going on in the world right now and given the risk that comes from distorted findings, we must ensure vigilance in how these studies are designed, conducted and judged," says lead investigator Michael Crocetti, M.D., M.P.H., a at Johns Hopkins Children's Center. "Our review is intended as a step in that direction."

Considered the gold standard of medical research, the hallmark of double-blind (RTC) is a design that rules out or accounts for actual or potential bias. Results of such studies, when peer-reviewed and published in reputable medical journals, can influence the practice of medicine and . A poorly designed or executed trial can therefore lead researchers to erroneous conclusions about the effectiveness of a drug or a procedure.

Citing the degree of bias risk in the studies they reviewed, the researchers caution pediatricians to be critical readers of studies, even in highly respected journals.

The investigators advise that when reading a report on a trial, pediatricians should not merely look at the bottom line but ask two essential questions: How did the researchers reach the conclusion? and Was their analysis unbiased?

Doctors should apply "smell tests," common sense and skeptical judgment about whether the conclusions fit the data, especially when a study boasts dramatic effects or drastic improvement.

Crocetti and colleagues used the Cochrane Collaboration tool, which assesses risk for bias along six critical aspects including randomization — randomly assigning patients to different treatments — and masking, the degree to which neither the patient nor the doctor knows which group of patients is receiving an active drug or intervention versus a placebo.

Investigators say that by analyzing each clinical trial along these and four other dimensions, the Cochrane Collaboration can answer what are perhaps the most important questions in medical research: How strong is the causal relationship between the therapy and the effect? and How valid are the conclusions made about the effect of the therapy?

Overall, 41 percent of the 146 trials in the review had improper or poorly described randomization techniques. Industry-funded trials were six times more likely to have high risk for biased randomization than government-funded trials or those funded by nonprofit organizations. And past research, the investigators point out, has shown that industry-funded trials are four to five times more likely to recommend an experimental drug.

"Industry funding is an important driver of medical discovery, but it is critical for investigators involved in such trials to ensure not only that the studies are conceived and executed cautiously with minimum risk for bias, but that any precautions taken against bias are also reported transparently," Crocetti says.

Trial registration — and the transparency commitment it reflects — is a key step in reducing bias or its influence, the researchers say. In their evaluation, registered trials were nearly 70 percent more likely to have robust randomization than non-registered trials, probably because the registration process itself forces researchers to answer many questions related to trial design and execution. Therefore, the investigators say, registering pediatric trials in the public domain will not only increase transparency but, in the long run, improve the validity of their results. In 2005, the International Committee of Medical Journal Editors called for registering all medical trials involving human subjects with www.ClinicalTrials.gov, a massive public repository of more than 70,000 trials from all over the world. However, less than 60 percent of the pediatric trials in the review were registered there.

The researchers also found that most of the trials (57 percent) either failed to use proper techniques that ensure anonymity or "blinding" to the type of treatment a patient gets, or they failed to clearly describe these techniques. The technique, called allocation concealment, ensures that neither the researcher nor the patient can guess which treatment they will get. The method also helps ensure that the treatment of one subject will not reveal to either scientists or the patients clues about the treatment of the next subject. Trials involving behavioral therapies were four times more likely to have this problem.

Overall, nearly 20 percent of the trials used improper masking techniques to ensure that neither the patient nor the researchers know which treatment went to which patient.

The 146 trials in the review appeared between 2007 and 2008 in the five leading pediatric journals — Pediatrics, Pediatric Infectious Diseases, Journal of Pediatrics, Archives of Pediatrics and Adolescent Medicine and the Journal of the American Academy of Child and Adolescent Psychiatry — and the three highest-ranked general medical journals — JAMA, The New England Journal of Medicine and The Lancet.

Explore further: Allergan to cut 1,500 employees in restructuring (Update)

More information: www.pediatrics.org

Provided by Johns Hopkins Medical Institutions

5 /5 (1 vote)
add to favorites email to friend print save as pdf

Related Stories

Extensive publication bias for Phase I drug trials

Feb 18, 2009

A study published in this week's issue of the open-access journal PLoS Medicine suggests that, in comparison to other types of trials, the results of Phase I drug trials are far less likely to be published.

Half of trials supporting FDA applications go unpublished

Sep 23, 2008

Over half of all supporting trials for FDA-approved drugs remained unpublished 5 years after approval, says new research published in this week's PLoS Medicine. The most important trials determining efficacy, and those with s ...

Doubts cast on credibility of some published clinical trials

Jul 02, 2009

Randomised Controlled Trials (RCTs) are considered the 'gold standard' research method for assessing new medical treatments. But research published in BioMed Central's open access journal Trials shows that the design of a r ...

Recommended for you

British Lords hold ten-hour debate on assisted dying

Jul 19, 2014

Members of Britain's unelected House of Lords spent almost ten hours on Friday discussing whether to legalise assisted dying, in an often emotional debate putting the question back on the agenda, if not on the statute books.

AbbVie, Shire agree on $55B combination

Jul 18, 2014

The drugmaker AbbVie has reached a deal worth roughly $55 billion to combine with British counterpart Shire and become the latest U.S. company to seek an overseas haven from tax rates back home.

Safety problems at US germ labs acknowledged

Jul 16, 2014

(AP)—The director of the U.S. Centers for Disease Control and Prevention acknowledged Wednesday that systemic safety problems have for years plagued federal public health laboratories that handle dangerous ...

User comments : 0