New proof reveals fundamental limits of scientific knowledge

May 3, 2018 by Jenna Marshall, Santa Fe Institute
A correct prediction (left), and an incorrect prediction (right). Illustration by Joerael Elliott, based on a diagram by David Wolpert. Credit: Santa Fe Institute

A new proof by SFI Professor David Wolpert sends a humbling message to would-be super intelligences: you can't know everything all the time.

The proof starts by mathematically formalizing the way an "inference device," say, a scientist armed with a supercomputer, fabulous experimental equipment, etc., can have knowledge about the state of the universe around them. Whether that scientist's knowledge is acquired by observing their universe, controlling it, predicting what will happen next, or inferring what happened in the past, there's a mathematical structure that restricts that knowledge. The key is that the inference device, their knowledge, and the physical variable that they (may) know something about, are all subsystems of the same universe. That coupling restricts what the device can know. In particular, Wolpert proves that there is always something that the inference device cannot predict, and something that they cannot remember, and something that they cannot observe.

"In some ways this formalism can be viewed as many different extensions of [Donald MacKay's] statement that 'a prediction concerning the narrator's future cannot account for the effect of the narrator's learning that prediction,'" Wolpert explains. "Perhaps the simplest extension is that, when we formalize [inference devices] mathematically, we notice that the same impossibility results that hold for predictions of the future—MacKay's concern—also hold for memories of the past. Time is an arbitrary variable—it plays no role in terms of differing states of the universe."

Not everyone can be right

What happens if we don't require that an inference device know everything about their universe, but only that it knows the most that could be known? Wolpert's mathematical framework shows that no two inference devices who both have free will (appropriately defined) and have maximal knowledge about the universe can co-exist in that universe. There may (or not) be one such "super inference device" in some given universe—but no more than one. Wolpert jokingly refers to this result as "the monotheism theorem," since while it does not forbid there being a deity in our universe, it forbids there being more than one.

As an example, suppose that Bob and Alice are both scientists with unlimited computational abilities. Moreover, suppose that they both have "free will," in that the question Bob asks himself does not restrict the possible questions Alice could ask herself, and vice-versa. (This turns out to be crucial.) Then it is impossible for Bob to predict (or retrodict) what Alice thinks at another time if Alice is also asked to predict what Bob is not thinking at that time.

New proof reveals fundamental limits of scientific knowledge
Credit: Santa Fe Institute

Wolpert compares this proposition to the Cretan liar's paradox, in which Epimenides of Knossos, a Cretan, famously stated "all Cretans are liars." Unlike Epimenides' statement though, which exposes the problem of systems that have the capability of self-reference, Wolpert's reasoning also applies to inference devices without that capability.

In addition, in Wolpert's formalism, the same scientist, considered at two different moments in time, is two different inference devices. So while it could be that some inference device is a "super inference " at one moment, they could not be so more than once. Again tongue in cheek, he refers to this as the "deism" theorem, since it allows there to be a deity that knows the most that could be known at the beginning of the universe—but forbids their ever being so knowledgeable again.

Because it does not rely on specific theories of physical reality like quantum mechanics or relativity, the new proof presents a broad set of limits for exploring the nature of scientific knowledge.

"None of these results limiting knowledge acquired by prediction relies on there being chaotic processes in the … it doesn't matter what the laws of physics are or if Alice is more computationally powerful than a Turing machine," says Wolpert. "All of this is independent of that and it's much broader ranging."

This research is progressing in many different directions, ranging from epistemic logic to a theory of Turing machines. In particular, Wolpert and his colleagues are creating a more nuanced, probabilistic framework that will allow them to explore not only the limits of absolutely correct , but also what happens when the inference devices are not required to know with 100% accuracy.

"What if Epimenides had said 'the probability that a Cretan is a liar is greater than x percent?'" Moving from impossibility to probability could tell us whether knowing one thing with greater certainty inherently limits the ability to know another thing. According to Wolpert, "we are getting some very intriguing results."

Explore further: Can scientists know that they do not know?

More information: Read David Wolpert's "Theories of Knowledge and Theories of Everything" chapter in the The Map and the Territory: www.springer.com/us/book/9783319724775

Constraints on physical reality arising from a formalization of knowledge: arXiv:1711.03499 [physics.hist-ph] arxiv.org/abs/1711.03499

Related Stories

Can scientists know that they do not know?

February 17, 2014

Imagine you knew everything about the current universe – the state of every single particle – and all the laws governing the universe's evolution. Endowed with such knowledge, you could then predict the future, right? ...

Study measures bias in how we learn and make decisions

April 26, 2016

Thinking about drawing to an inside straight or playing another longshot? Just remember that while human decision-making is biased by potential rewards, what we know about individual cues that help us to make those decisions ...

The man with the golden brain

December 13, 2011

What’s the point of a brain? A fundamental question that has led Professor Daniel Wolpert to some remarkable conclusions about how and why the brain controls and predicts movement. In a recent talk for TED, Wolpert explores ...

Working group: What physical principles predict life?

October 29, 2015

We are immersed in life here on Earth, but life isn't found on the Moon. Nor has it arisen, so far as we know, anywhere else in the solar system. Why do some physical environments precipitate life, and why don't others?

Recommended for you

19 comments

Adjust slider to filter visible comments by rank

Display comments: newest first

rgeContextomics
3.5 / 5 (2) May 03, 2018
Well duh! this always seemed self-obvious to me

No subsystem can mirror all the incident info propagated by a larger system that it is a subsystem of.

Is there a proof proving that the obvious is obvious?
Spaced out Engineer
1 / 5 (2) May 03, 2018
Their definition of "divine intervention," does not allow all of creation as one timeless event.
Monism has its problems too. Such a super inference device's field generation of knowledge, would be ambiguous.

An implicit hierarchy does not eliminate transfinite induction or pluralistic aggregation.
I still don't get how mutual information could not permit sharing the same space. What if all there was to know was the transaction protocol, or the noise from which is was derived?
Seems silly to say assume Laplace's demon because, we can't handle Cantor's diagonalization with the constraints of consistency.

Not being an expert however, I do agree that if you are going to have causal relations, and impose consistency, super determinism is the way to go.

It would be fascinating is logic alone said conditional probability was the wiser choice. Though unitarity and the Gaussian distribution form a seeming axiomatic basis for classical statistics.
Whydening Gyre
5 / 5 (1) May 03, 2018
Their definition of "divine intervention," does not allow all of creation as one timeless event.
Monism has its problems too. Such a super inference device's field generation of knowledge, would be ambiguous.

An implicit hierarchy does not eliminate transfinite induction or pluralistic aggregation.
I still don't get how mutual information could not permit sharing the same space. What if all there was to know was the transaction protocol, or the noise from which is was derived?
Seems silly to say assume Laplace's demon because, we can't handle Cantor's diagonalization with the constraints of consistency.
...
It would be fascinating is logic alone said conditional probability was the wiser choice. Though unitarity and the Gaussian distribution form a seeming axiomatic basis for classical statistics.

SOE - I'll have some of whatever you're smokin'....
Spaced out Engineer
1 / 5 (1) May 03, 2018
or the noise from which "it" was derived?
...
It would be fascinating "if" logic alone said conditional probability was the wiser choice. ...

SOE - I'll have some of whatever you're smokin'....

Ain't been shit... At least for a while now.
Sorry for the typos though. What I said still stands.
TheGhostofOtto1923
1.7 / 5 (3) May 04, 2018
I was already to trash this as it looked like it was from some philo but... its not. Only someone deluded into thinking that science has limits.
antialias_physorg
3.8 / 5 (4) May 04, 2018
At first the article sounded a lot like the restatement of Gödel's incompleteness theorem, but this part
which exposes the problem of systems that have the capability of self-reference, Wolpert's reasoning also applies to inference devices without that capability.

seems to indicate otherwise.

Though the proof still seems to hinge on there being something like a 100% correct fact (or "capital-T Truth" if you so will) - which I'm not at all sure can be argued in a watertight way.

(Note also that the 'monotheism' statement doesn't mean that there can be only one such super inference device...just not two at the same time. There could be one that later gets replaced by another)
knowphiself
2.3 / 5 (3) May 04, 2018
(seriously) Donald Rumsfeld (his three laws of knowing) makes more sense than that of an ivory tower mathematician's theorising / numerical musings : what is known in hind sight as mumbo jumbo. Or is it a way of just keeping grant money flowing. As to the laws of physics ; there was this one guy who could walk on water !
knowphiself
3 / 5 (2) May 04, 2018
Recognizing "spin" in the scientific literature
May 2, 2018 by Tasnim Elmamoun, Public Library of Science

Read more at: https://phys.org/...html#jCp
neiorah
1 / 5 (1) May 04, 2018
The reason is because we are not perfect and we cannot make anything perfect so this does not surpise me in the least.
Da Schneib
3.3 / 5 (3) May 04, 2018
Going by the article, this amazing "proof" shows that you can't find out what the water pressure is with a particle accelerator no matter how many computers you have.

I am unimpressed.
TheGhostofOtto1923
3.7 / 5 (3) May 05, 2018
The reason is because we are not perfect and we cannot make anything perfect so this does not surpise me in the least
'Perfect' is a word that has no definition. It was invented by religionists/philos to generate endless discussion about things that don't exist and don't have to exist.

It was meant to generate nonsense like this

"Philo of Alexandria in his first-century book "On the creation" mentions perfect numbers, claiming that the world was created in 6 days and the moon orbits in 28 days because 6 and 28 are perfect..."

-Philo - the first philo (probably not)
torbjorn_b_g_larsson
5 / 5 (1) May 05, 2018
Let us be clear that Wolpert is working with philosophy and not testable science, he does not attempt to provide tests that could reject his ideas but is looking at mathematical consistency. Predictably using it to criticize science only shows that the comment is not useful on science and can be downvoted.

That said, Wolpert's work was more interesting than I thought. It does not attempt to describe measurements or learning in general but assumes given knowledge. So primary finds is that "you cannot know what you cannot know" and that you can set up different instruments that conflicts in "what is to be known". When probability is introduced Wolpert's ideas becomes more realistic and the latter conflict disappears and consistent science knowledge is allowed. I do not understand Wolpert's claim that a finite universe cannot be fully known however, I think his proof then tries to infer knowledge that is not of the universe.
torbjorn_b_g_larsson
5 / 5 (1) May 05, 2018
[ctd]
Finally the model allows an analog to Hilbert's uncertainty principle (HIP) of statistical inferences; HIP is a deep result indeed! Though notably we now know that entanglement serves to undercut HIP due to the added system information so we can know more, Wolpert's model cannot describe that. Good luck with a formal axiomatic model describing science and its methods, re Gödel's results on what happens with sufficiently complex axiomatic models (inconsistencies)...
InquiringMind
4.5 / 5 (2) May 05, 2018
Gödel's proof showed any non-trivial theory expressed in a countable language is either inconsistent or incomplete. It uses a method based on a self-referential paradox. Gödel's result also "does not rely on specific theories of physical reality like quantum mechanics or relativity". It affects any theory that both includes basic arithmetic and could in principle be written down, so that includes any description of the universe in a human language.

Human comprehension may be similarly limited, though Wolpert is also concerned with non-human comprehension.

From this synopsis it's hard to be sure if Wolpert's method really does not appeal to self-reference. If not that would be a big achievement. However at least the Alice/Bob argument sounds to me suspiciously like self-reference generated in a new way. I look forward to learning more about it.
Da Schneib
1 / 5 (1) May 05, 2018
Errr, well I suppose that depends on how you define "non-trivial theory."
Whydening Gyre
not rated yet May 05, 2018
The reason is because we are not perfect and we cannot make anything perfect so this does not surpise me in the least.

I suppose it would depend on how much of a perfectionist you are....
Whydening Gyre
not rated yet May 05, 2018
The reason is because we are not perfect and we cannot make anything perfect so this does not surpise me in the least
'Perfect' is a word that has no definition. It was invented by religionists/philos to generate endless discussion about things that don't exist and don't have to exist.

It was meant to generate nonsense like this

"Philo of Alexandria in his first-century book "On the creation" mentions perfect numbers, claiming that the world was created in 6 days and the moon orbits in 28 days because 6 and 28 are perfect..."

-Philo - the first philo (probably not)

Well, he DID get a whole sophy named after him...
Whydening Gyre
not rated yet May 05, 2018
First sentence in the article;
'A new proof by SFI Professor David Wolpert sends a humbling message to would-be super intelligences: you can't know everything all the time."
By knowing this, you complete the circle..:-)
Shit changes too fast to be up on it all, all the time.
What's important is - you can learn and know about it if you have the time and inclination to keep up on - everything...

InquiringMind
not rated yet May 06, 2018
Errr, well I suppose that depends on how you define "non-trivial theory."

For the avoidance of doubt, in my comment above, "non-trivial theory." = any theory ìncluding basic arithmetic (Peano axioms or equivalent))

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.