News reports that don't report magnitude of scientific findings could mislead the public

When media coverage doesn't include the numerical magnitude of a scientific study's effect, the risk of readers developing biases increases significantly, according to a new University of Michigan study.

The study is published in the journal Psychological Science.

People may incorrectly assume that the findings are of greater importance and impact than they are without the numerical information from media outlets, researchers said.

"People often make everyday decisions based on science findings they read about in the media," said Audrey Michal, the study's lead author and assistant research scientist in the U-M Department of Psychology. "However, people might assume that scientific findings are more impactful than they truly are."

Michal, who co-authored the study with U-M psychology professor Priti Shah, said that when numerical information about the magnitude of a finding is absent, people assume that a finding is meaningfully large—what scientists refer to as "practically significant."

If people erroneously assume that science findings are practically significant, they may adopt ineffective health, dietary and other lifestyle interventions based on limited information, she said.

In contrast, transparently reporting the magnitude of findings can help people make better-informed everyday decisions. The new study involved the responses of 800 adults regarding interventions of different sizes.

People who read about the benefits of an expensive intervention (e.g., student math performance improvements from a new, costly math curriculum) of unreported magnitude ("Group A improved more than Group B") were more likely to endorse the intervention than those who read about a trivially-sized benefit (e.g., "Group A improved 2% more than Group B"). They were just as likely to endorse the intervention as those who read about a meaningfully large benefit (e.g., "Group A improved 10% more than Group B").

"Laypeople tended to assume that scientific findings had meaningfully large effects or were of high practical significance," Michal said. "Failing to report the magnitude of science findings is thus potentially misleading for the general public."

Overall, those who read about findings with a meaningfully large benefit were more likely to endorse the intervention than those who read about a trivially-sized benefit. Michal said this suggests that people can make better-informed decisions when effect magnitudes of science findings are reported in media reports.

However, participants with lower numeracy were more likely to endorse interventions with trivially-sized benefits than those with higher numeracy, suggesting that low-numerate individuals need help to understand whether the magnitude of scientific findings is meaningful.

Michal said science journalists can help low-numeracy individuals interpret the impact of study findings by making small numerical effects more concrete, or by providing additional context to help people understand the meaning of small numerical effects.