New proof reveals fundamental limits of scientific knowledge

May 3, 2018 by Jenna Marshall, Santa Fe Institute
A correct prediction (left), and an incorrect prediction (right). Illustration by Joerael Elliott, based on a diagram by David Wolpert. Credit: Santa Fe Institute

A new proof by SFI Professor David Wolpert sends a humbling message to would-be super intelligences: you can't know everything all the time.

The proof starts by mathematically formalizing the way an "inference device," say, a scientist armed with a supercomputer, fabulous experimental equipment, etc., can have knowledge about the state of the universe around them. Whether that scientist's knowledge is acquired by observing their universe, controlling it, predicting what will happen next, or inferring what happened in the past, there's a mathematical structure that restricts that knowledge. The key is that the inference device, their knowledge, and the physical variable that they (may) know something about, are all subsystems of the same universe. That coupling restricts what the device can know. In particular, Wolpert proves that there is always something that the inference device cannot predict, and something that they cannot remember, and something that they cannot observe.

"In some ways this formalism can be viewed as many different extensions of [Donald MacKay's] statement that 'a prediction concerning the narrator's future cannot account for the effect of the narrator's learning that prediction,'" Wolpert explains. "Perhaps the simplest extension is that, when we formalize [inference devices] mathematically, we notice that the same impossibility results that hold for predictions of the future—MacKay's concern—also hold for memories of the past. Time is an arbitrary variable—it plays no role in terms of differing states of the universe."

Not everyone can be right

What happens if we don't require that an inference device know everything about their universe, but only that it knows the most that could be known? Wolpert's mathematical framework shows that no two inference devices who both have free will (appropriately defined) and have maximal knowledge about the universe can co-exist in that universe. There may (or not) be one such "super inference device" in some given universe—but no more than one. Wolpert jokingly refers to this result as "the monotheism theorem," since while it does not forbid there being a deity in our universe, it forbids there being more than one.

As an example, suppose that Bob and Alice are both scientists with unlimited computational abilities. Moreover, suppose that they both have "free will," in that the question Bob asks himself does not restrict the possible questions Alice could ask herself, and vice-versa. (This turns out to be crucial.) Then it is impossible for Bob to predict (or retrodict) what Alice thinks at another time if Alice is also asked to predict what Bob is not thinking at that time.

New proof reveals fundamental limits of scientific knowledge
Credit: Santa Fe Institute

Wolpert compares this proposition to the Cretan liar's paradox, in which Epimenides of Knossos, a Cretan, famously stated "all Cretans are liars." Unlike Epimenides' statement though, which exposes the problem of systems that have the capability of self-reference, Wolpert's reasoning also applies to inference devices without that capability.

In addition, in Wolpert's formalism, the same scientist, considered at two different moments in time, is two different inference devices. So while it could be that some inference device is a "super inference " at one moment, they could not be so more than once. Again tongue in cheek, he refers to this as the "deism" theorem, since it allows there to be a deity that knows the most that could be known at the beginning of the universe—but forbids their ever being so knowledgeable again.

Because it does not rely on specific theories of physical reality like quantum mechanics or relativity, the new proof presents a broad set of limits for exploring the nature of scientific knowledge.

"None of these results limiting knowledge acquired by prediction relies on there being chaotic processes in the … it doesn't matter what the laws of physics are or if Alice is more computationally powerful than a Turing machine," says Wolpert. "All of this is independent of that and it's much broader ranging."

This research is progressing in many different directions, ranging from epistemic logic to a theory of Turing machines. In particular, Wolpert and his colleagues are creating a more nuanced, probabilistic framework that will allow them to explore not only the limits of absolutely correct , but also what happens when the inference devices are not required to know with 100% accuracy.

"What if Epimenides had said 'the probability that a Cretan is a liar is greater than x percent?'" Moving from impossibility to probability could tell us whether knowing one thing with greater certainty inherently limits the ability to know another thing. According to Wolpert, "we are getting some very intriguing results."

Explore further: Can scientists know that they do not know?

More information: Read David Wolpert's "Theories of Knowledge and Theories of Everything" chapter in the The Map and the Territory: www.springer.com/us/book/9783319724775

Constraints on physical reality arising from a formalization of knowledge: arXiv:1711.03499 [physics.hist-ph] arxiv.org/abs/1711.03499

Related Stories

Can scientists know that they do not know?

February 17, 2014

Imagine you knew everything about the current universe – the state of every single particle – and all the laws governing the universe's evolution. Endowed with such knowledge, you could then predict the future, right? ...

Study measures bias in how we learn and make decisions

April 26, 2016

Thinking about drawing to an inside straight or playing another longshot? Just remember that while human decision-making is biased by potential rewards, what we know about individual cues that help us to make those decisions ...

The man with the golden brain

December 13, 2011

What’s the point of a brain? A fundamental question that has led Professor Daniel Wolpert to some remarkable conclusions about how and why the brain controls and predicts movement. In a recent talk for TED, Wolpert explores ...

Working group: What physical principles predict life?

October 29, 2015

We are immersed in life here on Earth, but life isn't found on the Moon. Nor has it arisen, so far as we know, anywhere else in the solar system. Why do some physical environments precipitate life, and why don't others?

Recommended for you

How can you tell if a quantum memory is really quantum?

May 23, 2018

Quantum memories are devices that can store quantum information for a later time, which are usually implemented by storing and re-emitting photons with certain quantum states. But often it's difficult to tell whether a memory ...

Research reveals how order first appears in liquid crystals

May 22, 2018

Liquid crystals undergo a peculiar type of phase change. At a certain temperature, their cigar-shaped molecules go from a disordered jumble to a more orderly arrangement in which they all point more or less in the same direction. ...

27 comments

Adjust slider to filter visible comments by rank

Display comments: newest first

rgeContextomics
3.5 / 5 (2) May 03, 2018
Well duh! this always seemed self-obvious to me

No subsystem can mirror all the incident info propagated by a larger system that it is a subsystem of.

Is there a proof proving that the obvious is obvious?
mackita
1 / 5 (1) May 03, 2018
Is there a proof proving that the obvious is obvious?
This is the basis of information explosion: the contemporary science is drowning in research of increasingly trivial connections, the amount of which increases in geometric way. (1, 2, 3, 4, 5, 6,...) And once this information is not trivial, then it's at least irrelevant for future progress.
Spaced out Engineer
1 / 5 (2) May 03, 2018
Their definition of "divine intervention," does not allow all of creation as one timeless event.
Monism has its problems too. Such a super inference device's field generation of knowledge, would be ambiguous.

An implicit hierarchy does not eliminate transfinite induction or pluralistic aggregation.
I still don't get how mutual information could not permit sharing the same space. What if all there was to know was the transaction protocol, or the noise from which is was derived?
Seems silly to say assume Laplace's demon because, we can't handle Cantor's diagonalization with the constraints of consistency.

Not being an expert however, I do agree that if you are going to have causal relations, and impose consistency, super determinism is the way to go.

It would be fascinating is logic alone said conditional probability was the wiser choice. Though unitarity and the Gaussian distribution form a seeming axiomatic basis for classical statistics.
Whydening Gyre
5 / 5 (1) May 03, 2018
Their definition of "divine intervention," does not allow all of creation as one timeless event.
Monism has its problems too. Such a super inference device's field generation of knowledge, would be ambiguous.

An implicit hierarchy does not eliminate transfinite induction or pluralistic aggregation.
I still don't get how mutual information could not permit sharing the same space. What if all there was to know was the transaction protocol, or the noise from which is was derived?
Seems silly to say assume Laplace's demon because, we can't handle Cantor's diagonalization with the constraints of consistency.
...
It would be fascinating is logic alone said conditional probability was the wiser choice. Though unitarity and the Gaussian distribution form a seeming axiomatic basis for classical statistics.

SOE - I'll have some of whatever you're smokin'....
Spaced out Engineer
1 / 5 (1) May 03, 2018
or the noise from which "it" was derived?
...
It would be fascinating "if" logic alone said conditional probability was the wiser choice. ...

SOE - I'll have some of whatever you're smokin'....

Ain't been shit... At least for a while now.
Sorry for the typos though. What I said still stands.
TheGhostofOtto1923
2 / 5 (2) May 04, 2018
I was already to trash this as it looked like it was from some philo but... its not. Only someone deluded into thinking that science has limits.
mackita
1 / 5 (2) May 04, 2018
Science has indeed no limits in the sense of money spending - but it already exhibits serious limits in utilization of these money. For example the research becomes increasingly redundant, many things are invented again and again - and they're often forgotten first before they can be utilized. The contemporary research also becomes increasingly shallow and misleading.

Because the volume of information waiting for its reveal is infinite whereas the volume of money is always finite, we should think about ways, how to prioritize the research and to research primarily the stuff, which has immediate practical usage (like the overunity or cold fusion). This would give us enough of resources for research of this less usefull one.

Whereas the rules of contemporary science lead to quite opposite approach: the most abstract and least important things are researched first.
antialias_physorg
4.7 / 5 (3) May 04, 2018
At first the article sounded a lot like the restatement of Gödel's incompleteness theorem, but this part
which exposes the problem of systems that have the capability of self-reference, Wolpert's reasoning also applies to inference devices without that capability.

seems to indicate otherwise.

Though the proof still seems to hinge on there being something like a 100% correct fact (or "capital-T Truth" if you so will) - which I'm not at all sure can be argued in a watertight way.

(Note also that the 'monotheism' statement doesn't mean that there can be only one such super inference device...just not two at the same time. There could be one that later gets replaced by another)
knowphiself
2.3 / 5 (3) May 04, 2018
(seriously) Donald Rumsfeld (his three laws of knowing) makes more sense than that of an ivory tower mathematician's theorising / numerical musings : what is known in hind sight as mumbo jumbo. Or is it a way of just keeping grant money flowing. As to the laws of physics ; there was this one guy who could walk on water !
mackita
1 / 5 (2) May 04, 2018
How is Science Spun? Spin is a common tool of propaganda, often employed by media outlets to push an agenda. Within science, it is commonly used to generate 'science hype.' In these cases, the importance of scientific findings is overstated and relayed in an inappropriate manner. Spin has gained renewed interest by researchers as they have noticed that findings—sometimes preliminary ones—are being put forth in positive light so that they can be viewed favorably by the public. As Chiu and colleagues point out, spin can take a variety of forms, including:

1) "inappropriate study given study data;"
2) "inappropriate extrapolations or recommendations for clinical practice;"
3) "selective reporting;"
4) "more favorable data presentation"
knowphiself
3 / 5 (2) May 04, 2018
Recognizing "spin" in the scientific literature
May 2, 2018 by Tasnim Elmamoun, Public Library of Science

Read more at: https://phys.org/...html#jCp
neiorah
1 / 5 (1) May 04, 2018
The reason is because we are not perfect and we cannot make anything perfect so this does not surpise me in the least.
Da Schneib
4.5 / 5 (2) May 04, 2018
Going by the article, this amazing "proof" shows that you can't find out what the water pressure is with a particle accelerator no matter how many computers you have.

I am unimpressed.
TheGhostofOtto1923
5 / 5 (2) May 05, 2018
The reason is because we are not perfect and we cannot make anything perfect so this does not surpise me in the least
'Perfect' is a word that has no definition. It was invented by religionists/philos to generate endless discussion about things that don't exist and don't have to exist.

It was meant to generate nonsense like this

"Philo of Alexandria in his first-century book "On the creation" mentions perfect numbers, claiming that the world was created in 6 days and the moon orbits in 28 days because 6 and 28 are perfect..."

-Philo - the first philo (probably not)
torbjorn_b_g_larsson
5 / 5 (1) May 05, 2018
Let us be clear that Wolpert is working with philosophy and not testable science, he does not attempt to provide tests that could reject his ideas but is looking at mathematical consistency. Predictably using it to criticize science only shows that the comment is not useful on science and can be downvoted.

That said, Wolpert's work was more interesting than I thought. It does not attempt to describe measurements or learning in general but assumes given knowledge. So primary finds is that "you cannot know what you cannot know" and that you can set up different instruments that conflicts in "what is to be known". When probability is introduced Wolpert's ideas becomes more realistic and the latter conflict disappears and consistent science knowledge is allowed. I do not understand Wolpert's claim that a finite universe cannot be fully known however, I think his proof then tries to infer knowledge that is not of the universe.
torbjorn_b_g_larsson
5 / 5 (1) May 05, 2018
[ctd]
Finally the model allows an analog to Hilbert's uncertainty principle (HIP) of statistical inferences; HIP is a deep result indeed! Though notably we now know that entanglement serves to undercut HIP due to the added system information so we can know more, Wolpert's model cannot describe that. Good luck with a formal axiomatic model describing science and its methods, re Gödel's results on what happens with sufficiently complex axiomatic models (inconsistencies)...
InquiringMind
4.5 / 5 (2) May 05, 2018
Gödel's proof showed any non-trivial theory expressed in a countable language is either inconsistent or incomplete. It uses a method based on a self-referential paradox. Gödel's result also "does not rely on specific theories of physical reality like quantum mechanics or relativity". It affects any theory that both includes basic arithmetic and could in principle be written down, so that includes any description of the universe in a human language.

Human comprehension may be similarly limited, though Wolpert is also concerned with non-human comprehension.

From this synopsis it's hard to be sure if Wolpert's method really does not appeal to self-reference. If not that would be a big achievement. However at least the Alice/Bob argument sounds to me suspiciously like self-reference generated in a new way. I look forward to learning more about it.
Da Schneib
not rated yet May 05, 2018
Errr, well I suppose that depends on how you define "non-trivial theory."
Whydening Gyre
not rated yet May 05, 2018
The reason is because we are not perfect and we cannot make anything perfect so this does not surpise me in the least.

I suppose it would depend on how much of a perfectionist you are....
Whydening Gyre
not rated yet May 05, 2018
The reason is because we are not perfect and we cannot make anything perfect so this does not surpise me in the least
'Perfect' is a word that has no definition. It was invented by religionists/philos to generate endless discussion about things that don't exist and don't have to exist.

It was meant to generate nonsense like this

"Philo of Alexandria in his first-century book "On the creation" mentions perfect numbers, claiming that the world was created in 6 days and the moon orbits in 28 days because 6 and 28 are perfect..."

-Philo - the first philo (probably not)

Well, he DID get a whole sophy named after him...
Whydening Gyre
not rated yet May 05, 2018
First sentence in the article;
'A new proof by SFI Professor David Wolpert sends a humbling message to would-be super intelligences: you can't know everything all the time."
By knowing this, you complete the circle..:-)
Shit changes too fast to be up on it all, all the time.
What's important is - you can learn and know about it if you have the time and inclination to keep up on - everything...

mackita
5 / 5 (1) May 06, 2018
The problem is, it actually doesn't matter, which limits of information actually are in similar way, like it doesn't matter, how big the Universe actually is. Every system in this Universe would also have limited access to information, including the smallest unit of it: the individual (brains of) scientists itself. We simply have limited processing capacity and even collective decisions don't improve it very much: the collective stances gets fragmented anyway. It's clearly visible even on this quite tiny very forum.
In praxis the information follows the principle of least action and once the size of system across it is spreading grows too large, then it undergoes spontaneous symmetry breaking and it gets fragmented both is space but also both in time (it simply gets forgotten). The overall speed of progress will be thus limited by the speed in which individual members will be able to process new information - which will be the lower, the higher volume of info they already process.
mackita
5 / 5 (1) May 06, 2018
This process of information spreading across human civilization and scientific community in particular is analogous to freezing of energy spreading during condensation within multiparticle systems: once the system grows up to level, that the speed of energy spreading is not able to balance the forces acting between individual particles, then the system fragments itself into multiple crystals or droplets, which remain causally separated each other up to high degree.

In this context the reading of articles The era of expert failure by Arnold Kling, Why experts are usually wrong by David H. Freeman and Why the experts missed the crash by Phill Tetlock may be useful.
mackita
not rated yet May 06, 2018
The conclusion is, the more the volume of information grows, the more the individual people are forced to specialize, the more these experts will get separated from neighboring reality around them, this holistic one the more. For to keep the pace they should maintain very effective personal system for information sorting and prioritizing.
Another way, even more effective one is the prioritization of levels, at which information will be processed in a fractal way. It has again analogy in natural high dimensional systems, which evolved to handle energy very effectively.
It basically means, that some layer of experts will only deal with information at higher level (in form of reviews) and they will be authorized to say, what experts at lower levels will research and how. This approach is already anticipated in articles like this one.
mackita
not rated yet May 06, 2018
Of course many experts feel immediately offended by this proposal and they disagree hysterically, as they believe (Dunning-Krueger effect ensues) that just because they're most specialized, they should know best where to go at the general level. But the reality is, just the community of let say nuclear physicists dismisses the findings like the cold fusion the most - just because they're "best informed", why it should be "actually" impossible. In this case the investment into such an expertise becomes classical perverse incentive: the more financial and social support these experts will get, the more the actual process will be stalled.
mackita
not rated yet May 06, 2018
"Wolpert's mathematical framework shows that no two inference devices who both have free will (appropriately defined) and have maximal knowledge about the universe can co-exist in that universe"... - What the hell does "appropriately defined" mean in this context? If the device has contra-causal free will then it doesn't adhere to the rules of the universe. Is the quoted paragraph as ridiculous as it sounds?

Such an insight reflects common experience, that getting liberals to agree is like herding cats - they also like cats more for their independence than the conservatives. You cannot keep fully free will and to follow some consensus at the same moment - no matter how universally valid it actually is.
InquiringMind
not rated yet May 06, 2018
Errr, well I suppose that depends on how you define "non-trivial theory."

For the avoidance of doubt, in my comment above, "non-trivial theory." = any theory ìncluding basic arithmetic (Peano axioms or equivalent))

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.