Prediction or cause? Information theory may hold the key

September 30, 2011 by Miranda Marquit feature

( -- "A perplexing philosophical issue in science is the question of anticipation, or prediction, versus causality," Shawn Pethel tells "Can you tell the difference between something predicting an event and something actually causing an event?"

Pethel is a scientist working at the Redstone Arsenal in Alabama. Along with Daniel Hahs, he set out to identify a method of distinguishing anticipation from causality from information theory. “Any process that has to react in real time can improve its performance through anticipation, and in studying such processes it is important to find new ways to quantify causality” Pethel says.

The question of anticipation versus causality is one that has real-world application in a number of areas. Pethel points out that this issue has implications in covert operations, as well as in financial areas, especially with regard to the development of bubbles. “Is there a way, from passive measurements, to tell what’s driving what?” Pethel and Hahs try to answer this question in Physical Review Letters: “Distinguishing Anticipation from Causality: Anticipatory Bias in the Estimation of Information Flow.”

“If a system is generating information, it’s like a fingerprint that can be used to figure out where the information is coming from and where it is going,” Pethel explains. “A quantity called transfer entropy has been used since 1990 to measure information flow in experimental data from neuroscience, finance and even music.”

“What we were interested in,” he continues, “is the anticipation issue. We wanted to look at that using transfer entropy, and see if we could discover what’s predicting and what’s causing.”

In order to set up the situation, Pethel and Hahs used chaotic systems, which produce information. They were careful that their model would be connected in only one direction. “We set it up so that we know the causality – that x is causing y, but y cannot cause anything for x. However, we designed the y system to be able to predict x to some degree. We then collected data and used transfer to tell us which was the causal system.”

When analyzing the results, Pethel and Hahs found something rather interesting: Even though the response system, y, wasn’t the cause, it looked very much like it was under many different test conditions. “There is an anticipatory bias. It’s a very strong effect, the more anticipation there is, the stronger it will be. There is a huge bias going on in some cases, and it is giving the exact wrong answer. The system that was only predictive was indistinguishable from the cause.”

The good news is that Pethel and Hahs also noticed that there are some signs that indicate that the presence of a bias. “Even if you can’t pinpoint what the causal source is, you can see some behaviors that provide clues that you need to be on the alert for anticipatory dynamics.”

Pethel wants to take this further, though. “We’ve studied the one way system, and now that we know there is a bias, we can account for that, and study systems that have mutual back and forth coupling.”

Hopefully, the work will provide helpful clues in understanding how causality can be indentified using . “There’s a level of broad interest,” Pethel says, “since you can deduce causal mechanisms from data. Particularly interesting are financial markets and biological systems where anticipation plays a large role in allowing systems to adapt to a changing environment.”

Explore further: Don't have all the information? In the quantum world, that doesn't matter

More information: Daniel W. Hahs and Shawn D. Pethel, “Distinguishing Anticipation from Causality: Anticipatory Bias in the Estimation of Information Flow,” Physical Review Letters (2011). Available online:


Related Stories

The kids are alright

May 26, 2011

Children should be seen and not heard... who says? A Philosophy academic at The University of Nottingham is challenging the adage by teaching primary school children to argue properly.

Could primordial black holes be dark matter?

September 21, 2011

( -- “We know that about 25% of the matter in the universe is dark matter, but we don’t know what it is,” Michael Kesden tells “There are a number of different theories about what ...

Entanglement can help in classical communication

March 30, 2011

( -- When most of us think of entanglement, our minds jump immediately to quantum communication. "Entanglement has become very well known and useful in quantum communication," Robert Prevedel tells ...

Recommended for you

Black butterfly wings offer a model for better solar cells

October 19, 2017

(—A team of researchers with California Institute of Technology and the Karlsruh Institute of Technology has improved the efficiency of thin film solar cells by mimicking the architecture of rose butterfly wings. ...

New type of electron lens for next-generation colliders

October 18, 2017

Sending bunches of protons speeding around a circular particle collider to meet at one specific point is no easy feat. Many different collider components work keep proton beams on course—and to keep them from becoming unruly.


Adjust slider to filter visible comments by rank

Display comments: newest first

1 / 5 (3) Sep 30, 2011
Falls under a theory of fault-tolerant computing:;cad=rja

Anticipation plays a large role in fault-tolerance systems.
Better known under the poorer misnomers:back-up systems, redundancy, safety factors, etc.
4 / 5 (3) Sep 30, 2011
How can anything be reduced to a single cause ?
1 / 5 (5) Sep 30, 2011
There is no difference between the prediction as goal of the information technology in physics conceived as statistical anticipation of an incoming probable event and the principle of causality as conceived in deterministic physics. OPERA CNGS neutrino beam experiment (neutrinos could be faster than light!) has just revolutioned the meaning of 2-stroke causality that evolved in a 3-stroke tachionic causality. Some phoenomena in the universe could be anticipated by the emission of tachionic neutrinos (faster than light). A suddenly blowing supernova could be considered a 3-stroke event.
As 1-stroke the inertial observer watches tachionic neutrinos (anticipation as premonition)
As 2-stroke the i.o. watches the slower photonic emission of the blast (cause as lightning)
As 3-stroke the i.o. watches the sonic shockwave (effect as thunder). Therefore statistical premonition and deterministic premonition cold be the same thing while watching tachionic ruled events like supernovas.
1 / 5 (7) Sep 30, 2011
Can you tell the difference between something predicting an event and something actually causing an event
In dense aether model the difference between causality and consequence are losing their meaning at the sufficient distance from dimensional and energy density scale of human observer. It's similar to observation of water surface with its own ripples at the sufficient distance, when the surface ripples disperse into underwater waves completely. These underwater waves are spreading a much faster toward observer, so that every observer becomes both cause, both consequence of the remote events (compare the archetypal Ouroboros model of Universe of Mobius strip or Klein bottle topology).

From distant historical perspective of human society is often difficult to distinguish the cause from its consequences too. For example, was thinking of Einstein influenced with failure of sparse aether model, or did his thinking contribute to the failure of this model instead?
5 / 5 (1) Sep 30, 2011
Feedback Ojorf. Correlate the rating with words.
1 / 5 (7) Sep 30, 2011
How can anything be reduced to a single cause ?
There exist interesting one-electron universe hypothesis. This hypothesis has a good meaning in dense aether model.


In aether theory there exists one to many duality: for example, the behavior of stars is dual to behavior of atoms, from which these stars are mostly composed of, the counterpart of molecules are planets, the neutron stars correspond the atom nuclei, etc. At the very general level the Universe appears to have attributes of all visible categories inside of it, for example it resembles the interior of giant electron just with respect to its Mobius strip topology mentioned above and the electrons inside of it appear like holographic projection of this single electron into membranes of quantum foam forming the vacuum.
1 / 5 (5) Sep 30, 2011
Analogously, the particles of matter resemble the tiny black holes and the universe itself appears like the interior of black hole, the fluctuations of quantum foam are of hyperbolic topology in similar way, like the whole Universe by Roger Penrose. With compare to him astronomer Laura Mersini considers the whole universe as a single traveling quantum wave, i.e. the same artefact like every quantum particle, some others can see nested quantum branes or dodecahedrons or E8 gauge group in quantum foam geometry, the same topology can be applied to the observable Universe as a whole (ekpyrotic brane cosmology) - and we could definitely find many other examples of the one to many duality.
1 / 5 (1) Sep 30, 2011
there needs to be a better description (or maybe definition) of the cause vs effect. The way it looks now is both were set up to be similar but only one (effect) was measured, it showed the cause had matched it's results because of it's internal feedback. Why is that internal feedback necessary, it seems to goof up the result even if it wasn't connected. I apologize for recommending impossible math, but in '07 there was an article about the "non-locality of a single particle demonstrated without objection" (http://www.physor...784.html ), where they used a recursive split beam configuration (which I'm imagining can be duplicated mathematically) to obtain proof without affecting the cause. Perhaps that could be applied here.
5 / 5 (11) Sep 30, 2011
I think you all took this too too far...

Look, all he's saying is that sometimes it's hard to distinguish what is causing a problem, and what looks like its a cause, but is actually just another effect of the cause.

For example: Lets say for argument's sake, that the aurora borealis always happens two hours before the electrical grid goes down. Now, let's also assume that we do not have the technological or theoretical knowledge of solar flares.

One could assume that the aurora borealis CAUSES the electrical grid problems, when in fact it only PREDICTS the electrical grid going down.

This is important is that in many socialogical, economic, as well as certain bleeding edge cosmological and subatomic research - Where there are many variables/causes/predictors. It can help you determine if what you are looking at is not a cause.

In the comments we often say "Correlation does not mean causation". That's exactly why this research is so important.
1 / 5 (4) Sep 30, 2011
when in fact it only PREDICTS the electrical grid going down
Didn't you just want to say PRECEDES? IMO under the situation when some event always occurs before another one the determination of direction of the direction of their causality arrow is trivial.

The real question is the analysis of the causality arrow at the case of congruent events.
5 / 5 (5) Sep 30, 2011
No, I mean to say PREDICTS - as they used the term in the article.

It predicts by always preceding it in my example.

The real question is the analysis of the causality arrow at the case of congruent events.

at least you are getting somewhere in the general area of what this research is about. But congruent is not the right term.

You may have two correlated events - not necessarily congruent. You may have a feedback mechanism which may create a scenerio in which the cause actually looks like the effect, or a reverse correlation, etc.

You believe that one thing is the cause. This research is aimed at achieving a statistical analysis to help indicate if assumed cause is a correct cause. You may not even know the actual cause or causality arrow. But it will help rule out incorrect causes.
4.3 / 5 (3) Sep 30, 2011
The author wants a bridge from Kalman filtering to information theory. Fault-tolerance theory borrows from both. The physics display in the comments is OT.
5 / 5 (7) Sep 30, 2011
Hush is correct that the physics comments are off topic. Although this research can eventually be adapted and applied to cases of physics, it is not in any way about physics or causality on the quantum level.

This is a type of statistical analysis and information theory. You know what that is? Geek MATH.

Comments about physics only show a lack of comprehension or refusal to comprehend the article - ESPECIALLY SINCE THE ARTICLE SPECIFICALLY OMITTED THE STATISTICAL INDICATORS OF FALSE CAUSALITY. You cannot accurately apply this to anything without the key information.
1 / 5 (3) Sep 30, 2011
Comments about physics only show a lack of comprehension or refusal to comprehend the article
I commented the sentence from abstract "we show that in the low-resolution limit there is no statistic that can distinguish anticipatory elements from causal ones".

With using of the water surface model it can be interpreted like the "with increasing distance from observer we are unable to distinguish, whether the event is mediated with transverse or underwater waves (which actually heading toward us being faster than surface ripples, thus reversing the causality arrow of events)".

This theorem is actually of complex geometric nature. The fact it can be interpreted statistically shouldn't us detract from its effective meaning, because it deals with observable events, i.e. physical objects, not with abstract categories.
5 / 5 (3) Sep 30, 2011
Kudos, sir.

Hearing spans the entire realm - from physics of sound to statistical nature of the mind - never interpreting the same physical state with the same neuronal pathways associated with the physical. A weighted, filtered, statistical probability function.
5 / 5 (2) Sep 30, 2011
Your guns are well holstered in this thread. Obviously everyone knows who that is.
not rated yet Oct 01, 2011
At the singularity, where cause meets effect, aren't the two indistinguishable?
5 / 5 (4) Oct 01, 2011
Big thanks to Hush1 and 'that guy.' It is mathematical, and nothing more at the moment. I'll be interested to find what happens in the next stage of their work.

Callippo -- you're embarrassing yourself. Just FYI.
1 / 5 (1) Oct 02, 2011
Bayesian inference recognizes that the physical effect from cause is one-way, the arrow of time, but logical inference is not so constrained.
5 / 5 (1) Oct 02, 2011
What about Moores Law: Is it in fact a cause or an effect?
not rated yet Oct 02, 2011
Callippo -- you're embarrassing yourself. Just FYI.
People Who Embarrass Easily May Be More Trustworthy

What about Moores Law: Is it in fact a cause or an effect?
What about gravitational law? Is it a reason of why gravity is working - or just a numeric regression of observations?
5 / 5 (1) Oct 05, 2011
What about Moores Law: Is it in fact a cause or an effect?

Both. Computers are used to help design better computer chips, and we couldn't design chips that are this complex without them. However, it is also an effect of creating better manufacturing processes and the science behind it, which is helped by faster computers, but is dependant on brute science/research/building knowledge.

I still like your question though.
5 / 5 (1) Oct 06, 2011
What about gravitational law? Is it a reason of why gravity is working - or just a numeric regression of observations?- Calli

Here is the attempt at gravitation via Information Theory.

Here the goal here is a better(more complete) description of gravitation via Information Theory to improve the understanding between prediction and cause.

I still like your answers.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.