Confirmation bias in studies of gamma ray bursts

December 16, 2014 by David Coward, The Conversation
An artist’s conception of a of gamma ray burst. Credit: ESO/NAOJ

Our understanding of gamma ray bursts (GRBs) – flashes of gamma rays from explosions in distant galaxies – since they were discovered more than 50 years ago may not be as solid as first thought.

Research by myself and colleagues, published in Monthly Notices of the Royal Society Letters and a preprint, found that journals tended to publish papers which supported an already widely accepted model of how GRBs form, and reject those that don't – a form of confirmation bias.

But our work shows that results that don't conform to the standard model still need to be considered. So can the be salvaged in light of us uncovering the apparent confirmation bias?

NASA satellites detect an intense flash of gamma rays every few days, which are believed to originate from catastrophic explosions of (hypernovae) in galaxies at the edge of the known universe. From Earth, telescopes triggered by NASA usually detect a rapidly fading afterglow following the prompt short flash.

The most popular model that describes the physics of GRB afterglow creation is the standard model (not to be confused with the Standard Model of Particle Physics). One core prediction of the GRB model is that the total energy of the high-energy flash determines the brightness of the fading afterglow.

The standard model has been successful in describing many individual GRBs and has remained firmly entrenched as the favoured and only viable model to explain observations. It predicts a significant correlation between the total GRB energy emitted and afterglow luminosity.

It is what has been previously observed and reported in numerous published studies, adding further support to the standard model.

But is it the only model?

Other models

An alternative – the Cannonball Model – can explain several anomalous GRBs that don't fit the standard model. One important concern that applies to both models is that fine-tuning has to be done to make either model work for individual GRBs.

Since 2007 I had my doubts about the standard model as it seemed to fall short when I considered GRB properties of the whole population of several hundred.

My main issue was that many peer-reviewed GRB studies used the growing number of GRB distance estimates to determine how they are distributed throughout the universe. This is important (remembering GRBs result from massive stars exploding) because it provides a way to indirectly see how massive stars are distributed across the universe.

There is one problem with this idea: the distances are obtained from a rapidly fading optical afterglow to obtain spectroscopic redshift so many distances cannot be measured because the afterglow is too dim.

This causes a bias (known as the Malmquist Bias) to measure only the brightest and nearest GRBs. This implies that one preferentially selects some data while inadvertently removing other data.

Importantly, the Malmquist Bias then produces a "false" correlation between intrinsic properties, such as total energy and luminosity. I soon realised that the intrinsic properties of GRBs that relied on distance measurements were biased, but now had to prove this.

Rise to the challenge

In 2013, armed with my growing scepticism and new insight, I was determined to test the reported GRB correlations.

I led a small team that began analysing the published studies that showed significant correlations of intrinsic properties of GRBs (using the standard model). Although some of these works did attempt to remove biases, I soon realised an obvious one remained.

The GRB afterglow luminosity data was distance dependent, introducing a Malmquist bias, which was not corrected for!

After re-analysing the data, but removing the afterglow Malmquist Bias, I found that previous reported correlations could not be reproduced and are in fact non-existent. Furthermore, I found that by including only the biased data, the correlation returned.

What this really means is that many scientists ignored or did not see fundamental biases in the data. Our work showed that previously reported correlations are an artifact of observation – nothing to do with the intrinsic properties of GRBs.

How could this happen in a rigorous peer reviewed culture, and why did I see this and not anyone else?

The answer is rooted in the psychology of scientists participating in a relatively volatile field (for astronomy anyway) dominated by several strong collaborative groups. The GRB standard model has become a powerful dogma that cannot easily be challenged, especially with no viable alternative.

So the previous studies naturally interpreted the observed GRB correlations as a result of the standard model. In general terms this behaviour is termed confirmation bias (also called confirmatory bias or myside bias). It is a tendency for people to favour information that confirms their preconceptions or hypotheses regardless of whether the information is true.

Not perfect, but the best we have

From my personal account, it is understandable that the public may be sceptical of the scientific process. But ultimately the work we did to uncover the truth was vindicated by a rigorous peer review (not without setbacks, as we had to find scientists not connected with the GRB standard model) and now published results.

Despite the fact the scientific process is not perfect, it is the best system we have to converge to an informed understanding of the world around us. Sometimes this convergence is convoluted, as in my case, but ultimately science and society is a winner.

So what about the GRB standard model – is it in tatters? Actually, no. Its proponents are very good at fine-tuning the model to suit observations.

How much fine-tuning can you perform on a model before it should be abandoned? That is a question for another day.

Explore further: Gigantic explosions buried in dust: ALMA probes environment around dark gamma-ray bursts

Related Stories

Study of gamma-ray bursts afterglow surprises scientists

April 30, 2014

Research from an international team of scientists led by the University of Leicester has discovered for the first time that one of the most powerful events in our universe – Gamma-Ray Bursts (GRB) – behave differently ...

Astronomers reveal mystery of brightest ever gamma-ray burst

November 21, 2013

For the first time, a team of astronomers from around the world, including experts from the University of Leicester, have used data from satellites and observatories to explain the brightest Gamma-ray Burst (GRB) ever recorded.

The intergalactic medium in the young universe

September 30, 2013

(Phys.org) —In its earliest years, the universe was so hot that electrons and protons could not bind together in neutral atoms: all of the gas in the cosmos was ionized. Then, after 380,000 years of expansion, the universe ...

Recommended for you

Neutron-star merger yields new puzzle for astrophysicists

January 18, 2018

The afterglow from the distant neutron-star merger detected last August has continued to brighten - much to the surprise of astrophysicists studying the aftermath of the massive collision that took place about 138 million ...

New technique for finding life on Mars

January 18, 2018

Researchers demonstrate for the first time the potential of existing technology to directly detect and characterize life on Mars and other planets. The study, published in Frontiers in Microbiology, used miniaturized scientific ...

North, east, south, west: The many faces of Abell 1758

January 18, 2018

Resembling a swarm of flickering fireflies, this beautiful galaxy cluster glows intensely in the dark cosmos, accompanied by the myriad bright lights of foreground stars and swirling spiral galaxies. A1758N is a sub-cluster ...

13 comments

Adjust slider to filter visible comments by rank

Display comments: newest first

movementiseternal
Dec 16, 2014
This comment has been removed by a moderator.
imido
Dec 16, 2014
This comment has been removed by a moderator.
movementiseternal
Dec 16, 2014
This comment has been removed by a moderator.
movementiseternal
Dec 16, 2014
This comment has been removed by a moderator.
cantdrive85
1.7 / 5 (12) Dec 16, 2014
How could this happen in a rigorous peer reviewed culture, and why did I see this and not anyone else?


Yeah, how could this happen? Confirmation bias from scientists?

"The professional tends to interpret the pictures by using the theory he was taught while the amateur tries to use the picture to arrive at a theory" Halton Arp, Seeing Red

The answer is rooted in the psychology of scientists participating in a relatively volatile field (for astronomy anyway) dominated by several strong collaborative groups. The GRB standard model has become a powerful dogma...


Group think? Haven't herd that before...Not!

"The peer review system is satisfactory during quiescent times, but not during a revolution in a discipline such as astrophysics, when the establishment seeks to preserve the status quo." Hannes Alfvén

Tuxford
1.4 / 5 (11) Dec 16, 2014
'The GRB standard model has become a powerful dogma that cannot easily be challenged, especially with no viable alternative.'

....powerful dogma.... will be found one day to have detoured science into pursuing pointless dead ends, such as the Huge Bang Fantasy, non-existent gravity waves, and godless particles.
reset
1.7 / 5 (11) Dec 16, 2014
Ya gotta love an honest article...or..well...I do ( I can hear the indiscriminate mumbles of the ever vigilant Dogma gang here all asking "why feed the trolls with articles like this?"). Especially the part where the guy almost said the same thing I said to captain moderator regarding the peer review process.

"... as we had to find scientists not connected with the GRB standard model"

The scientists behind articles like this which admit the falability of the system deserve respect, and as he puts it, he was battling against a powerful dogma not easily challenged (he was talking about the people, not the data, which is why it passed peer review when you remove the PEOPLE who create the dogma).

Taking positive steps toward understanding is a refreshing change from the status quo.
Urgelt
3 / 5 (2) Dec 16, 2014
According to David Deutsche, proposed explanations which are 'easy to vary' are usually poor explanations.

It sounds to me as if the GRB standard model is easy to vary. If that's true, scientists should regard it as tentative and possibly incomplete, at best.
cantdrive85
1 / 5 (6) Dec 16, 2014
Most of the same downvoters, yet nothing to comment on?

https://sciencex....5/?v=act

If you ignore it, it will go away. The above statements apply to most of the theoretical sciences, astrophysics is the prime example.
Vietvet
5 / 5 (7) Dec 16, 2014
Most of the same downvoters, yet nothing to comment on?

https://sciencex....5/?v=act

If you ignore it, it will go away. The above statements apply to most of the theoretical sciences, astrophysics is the prime example.


There's no confirmation bias among Electric Universe devotees?
cantdrive85
1 / 5 (7) Dec 16, 2014
Most of the same downvoters, yet nothing to comment on?

https://sciencex....5/?v=act

If you ignore it, it will go away. The above statements apply to most of the theoretical sciences, astrophysics is the prime example.


There's no confirmation bias among Electric Universe devotees?

People are people, just as scientists are as well. Some here, you included, try to put science and scientists on a pedestal of infallibility.
Shootist
5 / 5 (5) Dec 16, 2014
Most of the same downvoters, yet nothing to comment on?

https://sciencex....5/?v=act

If you ignore it, it will go away. The above statements apply to most of the theoretical sciences, astrophysics is the prime example.


There's no confirmation bias among Electric Universe devotees?


Of course not, as there is nothing to confirm.
Captain Stumpy
5 / 5 (4) Dec 30, 2014
almost said the same thing I said to captain moderator
@resetTROLL
but you apply "confirmation bias" as well as conspiracy to everything

the only thing trustworthy to you, based upon your posts, is the pseudoscience posts of electrical engineers playing astrophysicist without any knowledge of all the aspects of astronomy, let alone the scale problem

i never said there were not problems with peer review OR the modern scientists (there are always problems with any fixed system because all fixed systems degrade over time)

- I said that it is the best system we have (or had) and that until there is something better which can weed out the idiocy of the pseudoscience, then we need to keep it

there is no worship there
there is no "science and scientists on a pedestal of infallibility" either

and what you DON'T get about the whole thing is the evidence part anyway

your failure
not mine

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.