Scientific evidence: What is it and how can we trust it?

The phrase "scientific evidence" has become part of the vernacular – thrown about like a hot potato during discussions of major environmental, health or social issues. Climate change is one example. The EU's ban on neonicotinoid pesticides is another.

We've heard numerous mentions of the associated "evidence", indicating the importance of the issue and the need for action. This evidence is presented by proponents in much the same way that evidence is given in a court case, usually to back up policies or decisions that will impact people's lifestyles. But, unlike in a court case, we are rarely told exactly where the evidence comes from and why it's evidence.

Scientific evidence is information gathered from scientific research, which takes a lot of time (and patience!) to conduct. But there are a few things that all this research needs to have in common to make it possible for decision-makers, and ultimately all of us, to accept it as "evidence".

Objective and unbiased

Research needs money to pay for laboratory equipment, field surveys, and materials – not to mention the wages of all the people involved in the project. And money certainly doesn't appear out of thin air, even around the quantum physics department!

The majority of researchers have to constantly apply for funds to carry out their research. These funds can come from different places, usually government bodies such as the Australian Research Council (ARC), academic or research institutions, non-profit organisations or even industry bodies. Applications are judged on scientific merit and their relevance to society or the funding body's interests.

Mostly, funds are distributed fairly. But if an organisation funds a research project that will benefit them financially, then we cannot accept the findings as "evidence" unless different researchers (from unrelated organisations) come to the same conclusions through their own independent research.

Ensuring results will be valid and accurate

Scientific evidence relies on data, and it is crucial for researchers to ensure that the data they collect is representative of the "true" situation. This means using proved or appropriate ways of collecting and analysing the data and ensuring the research is conducted ethically and safely.

Control scenarios may also be necessary when testing for effects or impacts – such as when developing new products (such as medicines), or evaluating management actions (such as farmland pesticide use). The control scenario represents the opposite of the scenario being tested. This is so the results that are seen in the test scenario are guaranteed to be from the tested product or impact, and nothing else.

If the scenario involves environmental processes of some kind, the test and control should ideally be carried out under natural conditions (or in an environment where these processes normally occur).

Sometimes this can be virtually impossible to do, and lab-based or combined lab/ field studies will need to be done instead so the "nuisance factors" can be controlled.

Take the recent neonicotinoid issue. If a researcher wants to prove that use of a pesticide does not affect bees flying about in the environment where the chemical is normally used, they will need to test two different scenarios.

One hive of bees will have to go about their business out in the field while being exposed to the pesticide. A second hive of bees will have to be in the same general environmental location as the first hive (to ensure both hives experience the same overall living conditions), but remain completely uncontaminated by the pesticide throughout the test.

It's obvious how impossible this would be to manage under natural conditions, where no one can control the drift of chemical droplets or the movement of tiny insects across the landscape! In this case, completely field-based studies may not exist, but it would be misleading to say that a "lack of field studies" means that the pesticide does not affect bees.

Peer-review and professional consensus

This step is the most crucial, and it turns research into the "evidence" that we all talk about. The researcher has to present their data, results and conclusions in the form of a scientific report or paper. This must be reviewed by their scientific peers – only they are qualified to assess the validity of the methods and the accuracy of the conclusions the researcher has drawn from the results.

Oscars are decided by international film industry professionals. Similarly, having research findings published in an international peer-reviewed journal essentially means that other professional scientists who specialise in that kind of research have verified the quality and validity of the research.

This process takes a long time – from submission of the manuscript to a journal, to the final publication date can take six months to a year, often longer.

For really important decisions, especially ones that will affect lots of people (how we should manage our national parks, for example), multiple studies may need to be sourced to show that a majority of scientists experienced with the issue agree on the evidence (just like a jury in a court case).

This is to show there is a "scientific consensus" on the evidence, and it provides even more reason for taking action on the issue at hand.

Of course, not everyone agrees on everything – think of any topic from the Earth being round, to what you and your family will eat for dinner tonight! So if a few scientists disagree with the majority group of scientists over a particular issue, that is not immediate proof that the evidence is wrong, and neither is it shocking or newsworthy.

After all, we don't talk about the Best Actor Academy Award nominee who got the fewest votes.

Interpreting the evidence presented to us

Most of us hear of "scientific evidence" from journalists, newsreaders, politicians or media commentators, and often we don't have the opportunity to check the facts ourselves. But understanding where true scientific evidence comes from, and what it means, is imperative to helping us tackle the most important issues affecting our own lives and the world we live in.

So the next time someone says they have "scientific evidence" to back up their case, ask a few questions. Who funded the research and why? How much evidence is there and how was it gathered? Was the sample size or location representative of the "real" situation?

Has the research been published in an internationally-accepted, peer-reviewed journal, or is it only available online on a personal or organisation's website? Do a majority of other scientists agree on these results? If a few disagree, are they qualified to evaluate the issue? (For example, a medical doctor and an astronomer are both scientists – but that doesn't mean the astronomer is qualified to perform heart surgery!)

And if someone claims there is a "lack" of evidence on a contested issue, ask them to clarify. Do they mean that peer-reviewed research has been carried out, and found no proof of an effect? Or, do they mean that no one has yet funded research to examine the issue? These do not mean the same thing – as the saying goes:

Absence of evidence is not evidence of absence!

This story is published courtesy of The Conversation (under Creative Commons-Attribution/No derivatives).