Clinical trials cited for ignoring previous relevant research

January 21, 2011

The vast majority of already published and relevant clinical trials of a given drug, device or procedure are routinely ignored by scientists conducting new research on the same topic, a new Johns Hopkins study suggests.

The vast majority of already published and relevant clinical trials of a given drug, device or procedure are routinely ignored by scientists conducting new research on the same topic, a new Johns Hopkins study suggests.

The authors of the findings, reported in the Jan. 4 issue of , argue that these omissions potentially skew scientific results, waste taxpayer money on redundant studies and involve patients in unnecessary research.

Conducting an analysis of published studies, the Johns Hopkins team concludes that researchers, on average, cited less than 21 percent of previously published, relevant studies in their papers. For papers with at least five prior publications available for , one-quarter cited only one previous trial, while another quarter cited no other previous trials on the topic. Those statistics stayed roughly the same even as the number of papers available for citation increased. Larger studies were no more likely to be cited than smaller ones.

“The extent of the discrepancy between the existing evidence and what was cited is pretty large and pretty striking,” says Karen Robinson, Ph.D., an assistant professor of medicine at the Johns Hopkins University School of Medicine and co-author of the research with Steven N. Goodman, M.D., M.H.S., Ph.D., a Hopkins epidemiology and biostatistics professor. “It’s like listening to one witness as opposed to the other 12 witnesses in a criminal trial and making a decision without all the evidence. Clinical trials should not be started — and cannot be interpreted — without a full accounting of the existing evidence.”

Robinson and Goodman searched the Web of Science, an Internet archive, for meta-analyses done in 2004 on groups of randomized, controlled trials on such common topics as a cancer treatment or a heart procedure. A meta-analysis is a systematic procedure for statistically combining the results of many different studies on a similar topic to determine the effectiveness of medical interventions.

The researchers ultimately looked at 227 meta-analyses comprising 1,523 separate clinical trials in 19 different fields, including oncology, neurology and pediatrics.

Of 1,101 peer-reviewed publications for which there had been at least five previous relevant papers, 46 percent acknowledged the existence of no more than one previous trial.

“Accurate representation of the prior cumulative evidence is necessary to both ethically justify a trial and to make proper inferences,” they write. Studying prior research can also lead to study designs that are more likely to fill gaps in the evidence, the team said, noting that although the presence of a citation “does not tell us how information from that trial was used, the absence of a citation almost guarantees that it was not.”

The Hopkins researchers could not say why prior trials failed to be cited or whether non-cited trials may have been taken into account in the trial design and proposal stages, such as grant requests to the National Institutes of Health.

At the very least, Robinson says, researchers often contend that their publications are so “unique” that there are no relevant studies to cite, even though someone else may have included it in a meta-analysis of like research. Others claim there just isn’t room to cite past relevant research, but Robinson says one reason for the omissions could be the self-interest of researchers trying to get ahead.

“To get published, journals are looking for novelty, uniqueness,” she says. Leaving out references to prior similar research can make findings seem more like a breakthrough, she adds. In her publications study, Robinson found several papers that claimed to be the first even when many trials on the subject preceded them.

There are no barriers to funding, conducting or publishing a clinical trial without proof that prior literature had been adequately searched and evaluated, she says. But requirements such as those have been instituted by some European funding agencies, the medical journal The Lancet, and the U.S. Centers for Medicare & Medicaid Services, which require that a covered trial not “unjustifiably duplicate existing studies,” Robinson writes.

Robinson says funders, institutional review boards and journals need to take steps to ensure that prior research is considered. To do otherwise, she says, encourages this “unethical” behavior to continue.

“Trials being done may not be justified, because researchers are not looking at or at least not reporting what is already known,” she says. “We may be wasting resources when we fund trials for which we already know the answer. And we may be coming to incorrect conclusions about what works in medicine.”

In some cases, patients who volunteer for may be getting a placebo for a medication that a previous researcher has already determined works or may be getting a treatment that another researcher has shown is of no value. In rare instances, patients have suffered severe side effects and even died in studies because researchers were not aware of previous studies documenting a treatment’s dangers.

Explore further: Telehealth keeps asthmatics out of hospitals

Related Stories

Telehealth keeps asthmatics out of hospitals

October 6, 2010

Health care delivered via telephone or Internet might not improve the quality of life for people with mild asthma, but it could keep those with severe asthma out of the hospital, a new evidence review finds.

Evidence of ginseng-boosted brain power is weak

December 8, 2010

Many people believe that the popular herb ginseng can improve thinking ability and prevent or even treat dementia. However, a comprehensive review of research failed to find convincing evidence of these benefits.

Mechanical versus manual CPR may be too close to call

January 19, 2011

Pushing on the chest to simulate the heart’s rhythmic pumping action is an essential part of cardiopulmonary resuscitation after cardiac arrest. In recent decades, manufacturers have developed several mechanical devices ...

Recommended for you

How the finch changes its tune

August 3, 2015

Like top musicians, songbirds train from a young age to weed out errors and trim variability from their songs, ultimately becoming consistent and reliable performers. But as with human musicians, even the best are not machines. ...

Machine Translates Thoughts into Speech in Real Time

December 21, 2009

( -- By implanting an electrode into the brain of a person with locked-in syndrome, scientists have demonstrated how to wirelessly transmit neural signals to a speech synthesizer. The "thought-to-speech" process ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Jan 21, 2011

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.