To create a super-intelligent machine, start with an equation

Nov 28, 2013 by Marcus Hutter, The Conversation
Where do you start building a machine like Deep Thought, the super-intelligent computer who came up with the Answer to The Ultimate Question of Life, the Universe, and Everything?

Intelligence is a very difficult concept and, until recently, no one has succeeded in giving it a satisfactory formal definition.

Most researchers have given up grappling with the notion of intelligence in full generality, and instead focus on related but more limited concepts – but I argue that mathematically defining intelligence is not only possible, but crucial to understanding and developing super-intelligent machines.

From this, my research group has even successfully developed software that can learn to play Pac-Man from scratch.

Let me explain – but first, we need to define "intelligence".

So what is intelligence?

I have worked on the question of general rational intelligence for many years. My group has sifted through the psychology, philosophy and literature and searched for definitions individual researchers and groups came up with.

The characterisations are very diverse, but there seems to be a recurrent theme which we have aggregated and distilled into the following definition:

Intelligence is an agent's ability to achieve goals or succeed in a wide range of environments.

You may be surprised or sceptical and ask how this, or any other single sentence, can capture the complexity of intelligence. There are two answers to this question:

1. Other aspects of intelligence are implicit in this definition: if I want to succeed in a complex world or achieve difficult goals, I need to acquire new knowledge, learn, reason logically and inductively, generalise, recognise patterns, plan, have conversations, survive, and most other traits usually associated with intelligence.

2. The challenge is to transform this verbal definition consisting of just a couple of words into meaningful equations and analyse them.

This is what I have been working on in the past 15 years. In the words of American mathematician Clifford A. Truesdell:

There is nothing that can be said by mathematical symbols and relations which cannot also be said by words. The converse, however, is false. Much that can be and is said by words cannot be put into equations – because it is nonsense.

Indeed, I actually first developed the equations and later we converted them into English.

Universal artificial intelligence

This scientific field is called universal artificial intelligence, with AIXI being the resulting super-intelligent agent.

The following equation formalises the informal definition of intelligence, namely an agent's ability to succeed or achieve goals in a wide range of environments:

Explaining every single part of the equation would constitute a whole other article (or book!), but the intuition behind it is as follows: AIXI has a planning component and a learning component.

Imagine a robot walking around in the environment. Initially it has little or no knowledge about the world, but acquires information from the world from its sensors and constructs an approximate model of how the world works.

It does that using very powerful general theories on how to learn a model from data from arbitrarily complex situations. This theory is rooted in algorithmic information theory, where the basic idea is to search for the simplest model which describes your data.

The model is not perfect but is continuously updated. New observations allow AIXI to improve its world model, which over time gets better and better. This is the learning component.

AIXI now uses this model for approximately predicting the future and bases its decisions on these tentative forecasts. AIXI contemplates possible future behaviour: "If I do this action, followed by that action, etc, this or that will (un)likely happen, which could be good or bad. And if I do this other action sequence, it may be better or worse."

The "only" thing AIXI has to do is to take from among the contemplated future action sequences the best according to the learnt model, where "good/bad/best" refers to the goal-seeking or succeeding part of the definition: AIXI gets occasional rewards, which could come from a (human) teacher, be built in (such as high/low battery level is good/bad, finding water on Mars is good, tumbling over is bad) or from universal goals such as seeking new knowledge.

The goal of AIXI is to maximise its reward over its lifetime – that's the planning part.

In summary, every interaction cycle consists of observation, learning, prediction, planning, decision, action and reward, followed by the next cycle.

If you're interested in exploring further, AIXI integrates numerous philosophical, computational and statistical principles:

Theory and practice of universal artificial intelligence

The above equation rigorously and uniquely defines a super-intelligent agent that learns to act optimally in arbitrary unknown environments. One can prove amazing properties of this agent – in fact, one can prove that in a certain sense AIXI is the most intelligent system possible.

Note that this is a rather coarse translation and aggregation of the mathematical theorems into words, but that is the essence.

Since AIXI is incomputable, it has to be approximated in practice. In recent years, we have developed various approximations, ranging from provably optimal to practically feasible algorithms.

At the moment we are at a toy stage: the approximation can learn to play Pac-Man, TicTacToe, Kuhn Poker and some other games.

This video is not supported by your browser at this time.
Watch AIXI play Pac-Man.

The point is not that AIXI is able to play these games (they are not hard) – the remarkable fact is that a single agent can learn autonomously this wide variety of environments.

AIXI is given no prior knowledge about these games; it is not even told the rules of the games!

It starts as a blank canvas, and just by interacting with these environments, it figures out what is going on and learns how to behave well. This is the really impressive feature of AIXI and its main difference to most other projects.

Even though IBM Deep Blue plays better chess than human Grand Masters, it was specifically designed to do so and cannot play Jeopardy. Conversely, IBM Watson beats humans in Jeopardy but cannot play chess – not even TicTacToe or Pac-Man.

AIXI is not tailored to any particular application. If you interface it with any problem, it will learn to act well and indeed optimally.

The current approximations are, of course, very limited. For the learning component we use standard file compression algorithms (learning and compression are closely related problems). For the planning component we use standard Monte Carlo (random search) algorithms.

Neither component has any particular built-in domain knowledge (such as the Pac-Man board or TicTacToe rules).

Of course you have to interface AIXI with the game so that it can observe the board or screen and act on it, and you have to reward it for winning TicTacToe or eating a food pellet in Pac-Man … but everything else AIXI figures out by itself.

Explore further: Angry Birds: The rise of the machines

More information: This article is adapted from a presentation which will be delivered at the Science, Technology and the Future conference, November 30 and December 1 2013.

Related Stories

Angry Birds: The rise of the machines

Oct 24, 2013

An EPFL team has won an international artificial intelligence competition with software able to reproduce a human-like way of playing the famous video game.

On the hunt for universal intelligence

Jan 27, 2011

How do you use a scientific method to measure the intelligence of a human being, an animal, a machine or an extra-terrestrial? So far this has not been possible, but a team of Spanish and Australian researchers ...

Recommended for you

Researchers developing algorithms to detect fake reviews

Oct 21, 2014

Anyone who has conducted business online—from booking a hotel to buying a book to finding a new dentist or selling their wares—has come across reviews of said products and services. Chances are they've also encountered ...

User comments : 29

Adjust slider to filter visible comments by rank

Display comments: newest first

grondilu
1 / 5 (1) Nov 28, 2013
to be removed please.
dogbert
1.3 / 5 (17) Nov 28, 2013
Impressive result.

Intelligence is an agent's ability to achieve goals or succeed in a wide range of environments.


The above definition would apply to a simple virus. I don't think intelligence can be so simply defined.
antialias_physorg
4 / 5 (6) Nov 28, 2013
The above definition would apply to a simple virus

Any definition for 'goal' you can find will use words like 'desire', 'endeavour', 'plan', 'object of ambition', etc. (i.e. some IDEA about a future state that is preferable over any alternate state)
None of these apply to a virus. Viruses don't have ideas.
Eikka
1.7 / 5 (17) Nov 28, 2013
Intelligence is an agent's ability to achieve goals or succeed in a wide range of environments.


That is an incomplete definition of intelligence, because it doesn't do away with the Turing's problem of the dumb machine being sufficiently complex to exhaust our ability to test it. In other words, we would not be able to tell the difference, so we would have to declare the agent intelligent even when it's not.

Intelligence is more properly defined as an agent's ability to produce succesful solutions autonomically beyond its internal mechanism. This excludes any version of intelligence that could in principle be performed by a simple automaton, as well as any fully deterministic program versions of intelligence.

Deterministic intelligence is not a meaningful concept since you cannot separate what the agent is doing from what its environment is doing and you are therefore unable to answer what exactly is the intelligent agent you're talking of.
Eikka
1.5 / 5 (17) Nov 28, 2013
Any definition for 'goal' you can find will use words like 'desire', 'endeavour', 'plan', 'object of ambition', etc.


Those are just synonyms.

They mean the same thing: a system's propensity towards some other state whether it gets there or not. If you say you have a desire or a goal to do something, you're simply saying that you're inclined to achieve it. If you don't have that tendency, then you were simply lying.

None of these apply to a virus. Viruses don't have ideas.


They don't need to in order to have a tendency to do something.
Eikka
1.3 / 5 (14) Nov 28, 2013
i.e. some IDEA about a future state that is preferable over any alternate state


More precisely, what do you think such an idea is?

A rock set on an incline has the implicit "idea" to roll down to an alternate state, because such ideas are nothing but information about where you are now, and where you want to be later.

Even a rock can have many such "ideas". If you have a rock that is shaped like a bowl and you place it on top of a upright stick, one way it will stay there and the other way it wants to fall off.

"Want" is the keyword in this question. What makes the "want" of an intelligent agent different from the tendecy of a rock or a virus to act like they do? If you cannot solve that, then there is no intelligence in the first place and the whole thing is just nonsense.
PsycheOne
1 / 5 (12) Nov 28, 2013
There are many definitions of intelligence. This article posits an agent that learns to play computer games. Regardless of the definition, it seems to me anything that can do that will fit any reasonable definition of intelligence, including our own intuitive perception of intelligence.
kochevnik
1 / 5 (13) Nov 28, 2013
This definition of intelligence overlooks the ability of higher organisms to dynamically define their reward, and dualistically. For example children play work and as adults work to play. Any system that cannot incorporate dialectics will not have "common sense" which is won by bridging intransitive relationships by experience
VendicarE
3.7 / 5 (3) Nov 28, 2013
"For the learning component we use standard file compression algorithms (learning and compression are closely related problems)."

Needs repeating for those who missed it.
VendicarE
3.3 / 5 (3) Nov 28, 2013
Whydening Gyre
1.3 / 5 (13) Nov 28, 2013
Goal oriented "intuitive leaps" using pattern observations to make prediction of pattern similarities in other environments so as to modify action would be an apt description of Intelligence, I would think...
Einstein was right - The only real valuable thing is Intuition...
Telekinetic
1 / 5 (12) Nov 28, 2013
Let's say AIXI's girlfriend finds a text from another computer program in AIXI's message bank. When confronted by his enraged girlfriend who by now is threatening to twist his transistors off, will he have the wherewithal to deny that he's guilty of any wrongdoing whatsoever, or will he cave in to his preprogrammed propensity for truth-telling? If he's able to deny convincingly, that's evidence of intelligence.
jimbo92107
1.4 / 5 (11) Nov 29, 2013
It's hard to imagine a machine intelligence that becomes an integral part of a mutually supportive social network. Siri is not intelligent, it is a clever pile of algorithms.

If you reward the machine for success, do you also punish the machine for failure? Who decides what is success or failure? How do you teach a machine to purposely lose a chess game to a little kid, so the kid can be encouraged? How do you teach a machine that hacking into a bank's network is bad?

How do you even define intelligence when our species has not settled on a consistent way of manifesting it?
Scryer
1 / 5 (1) Nov 29, 2013
What would be interesting is to see if you could confuse the algorithm by giving it a reward, and a demerit for the same thing once it's 'learned' what's 'right' and what's 'not'. Plenty of algorithms that play games have been 'brute forced' (properly researched) such as http://www.cs.cmu...7/mario/ to some great success.

What I wonder though is if it's possible to specify an algorithm that is aware of itself while also being aware of whatever you're trying to teach it or give it to learn. Maybe that's too abstract to really program. Anyway, very interesting. I know there's only so much that can be done with code, but I do believe we're getting insanely close to 'conscious' computers.

What's next is clearly some kind of bio-organic computer, new and exciting stuff.
nowhere
1 / 5 (1) Nov 29, 2013
agent's ability to produce succesful solutions autonomically beyond its internal mechanism.

This would imply humans arrive at solutions using something other than their internal mechanism. What is this extra something?
nowhere
3 / 5 (2) Nov 29, 2013
Impressive result.

Intelligence is an agent's ability to achieve goals or succeed in a wide range of environments.


The above definition would apply to a simple virus.

Of course it would, as it's a measure of intelligence of a living agent. How it scores at that measure defines it's intelligence. A virus would score extremely low as it can only achieve few goals in a limited environment, but a virus is still more intelligent than a rock. A dog would score better as it can achieve more goals in a variety of environments. A human would score exceedingly well by being able to achieve vasly more goals in exceedingly complex environments.
dogbert
1 / 5 (12) Nov 29, 2013
nowhere,

At a minimum, intelligence requires the ability to think. Since a virus has no ability to think nor mechanism capable of thought, it cannot be reasonably called intelligent. It does, however, have the ability to succeed in a range of environments.
nowhere
3.7 / 5 (3) Nov 29, 2013
At a minimum, intelligence requires the ability to think.

No, self awareness requires the ability to think. Intelligence is a measurement as defined in the article and, while being affected positively by the ability to think, doesn't not depend on it.

Since a virus has no ability to think nor mechanism capable of thought, it cannot be reasonably called intelligent.

No the virus's lack of a thought mechanism only severely limits its intelligence, since lack of thought negatively impacts its ability to succeed. Since the articles definition of intelligence is a measurement, it's not quite right to call the virus either "intelligent" or "not intelligent", but rather to find its level of intelligence relative to the range.
machinephilosophy
1 / 5 (12) Nov 29, 2013
As with both philosophy and science, there is here an avoidance of questions about ultimate criteria for determining what counts as intelligence.

However, if the criteria for determining what intelligence is are made explicit, then that criteria is necessarily being used as a meta-intelligence for making that determination.

Same thing occurs with regard to the God debate: the criteria for analyzing the issue is itself already God-level.
Telekinetic
1.3 / 5 (13) Nov 30, 2013
The whole premise is nullified unless consciousness accompanies intelligence. A program that is meant to be autonomous must make choices based not only on problem and puzzle solving but on instinct and feeling that transcend electronics. It's possible those unique characteristics belonging to living beings of the animal kingdom can be imitated, but machines will always be devoid of impulses like love, fear, rage, joy, etc. It's pure adolescent fantasy that this proves that life can be replicated by algorithms.
krundoloss
1 / 5 (11) Nov 30, 2013
The whole premise is nullified unless consciousness accompanies intelligence. A program that is meant to be autonomous must make choices based not only on problem and puzzle solving but on instinct and feeling that transcend electronics. It's possible those unique characteristics belonging to living beings of the animal kingdom can be imitated, but machines will always be devoid of impulses like love, fear, rage, joy, etc. It's pure adolescent fantasy that this proves that life can be replicated by algorithms.


If a small robot, with several senses, using the AIXI algorithm from the article, was constantly gathering information from its environment and running the algorithm against that data, is that not considered "thought"? If this algorithm does what they say, you could build a robot that will become a basic intelligence, with its own set of experience, conclusions and maybe, ideas.
Protoplasmix
1 / 5 (12) Nov 30, 2013
but I argue that mathematically defining intelligence is not only possible, but crucial to understanding and developing super-intelligent machines.

Most intelligent work! I'd say genius work :)

Intelligence certainly figures prominently in most formulas for human success. <-- The math inherent in that statement should be easy to recognize.

There is usually a single most intelligent way of doing something, and then there less intelligent ways, and then there is comic relief, or as VendicarE has aptly pointed out on many occasions, there are retarded ways of doing something. Easy enough to apply a rewards algorithm in all cases.
Egleton
1 / 5 (10) Dec 01, 2013
Does the program rely on push from the past or pull from the future?
If it relies on Pull from a future Attractor it will more closely resemble biological intelligence.
Kedas
1 / 5 (1) Dec 02, 2013
Intelligence is just the name for a certain amount of life of something compared to yours.
Obviously 'life' is not true or false but is a certain quantity of X, (I'm not going to tell what X is)
antialias_physorg
5 / 5 (2) Dec 02, 2013
I'm not going to tell what X is

Lemme guess: because you have no clue what you're talking about?

Does the program rely on push from the past or pull from the future?
If you want to word it that way: inteligence relies on a future attractzor (expected outcome). In lieu of that there was a recent article on here where intelligent behavior was defined as one that maximizes (apparent) future choices ... which I think is intrigueing definition as the only cases where I can come up with a counterexample the more limited choice/victory choice-path is confined due to externally imposed rules, or instincts. But if one were to go for a 'free' intelligence then the "maximum choice axiom" seems reasonable
Kedas
1 / 5 (1) Dec 04, 2013
I'm not going to tell what X is

Lemme guess: because you have no clue what you're talking about?

No, because I think only those that can figure it out should know. (and I'm not 100% sure yet that I'm right, haven't found any reason to prove that I'm wrong though)
Knowing is a bit like knowing the end of the joke before they tel you the joke the first time, you may choose not to know.
Mike_Massen
1 / 5 (4) Dec 04, 2013
It seems to me a key issue about intelligence (in respect of human capacity), of which some contend there are 7 or so distinct types (eg emotional, linguistic, spatial etc), is the ability to adapt multiple behaviours in depth in conjunction with combinatorial complexity of decision tree exploration on current tasks & particularly in relation to planning which does not necessarily have an immediate operant conditioned outcome ie positive/negative re-inforcement.

In respect of the above one might also consider how much we crave comfort by unconsciously contriving to manipulate circumstances to be a static, ostensibly to conserve effort (energy) vs appreciation of the dynamic which can release a flood of options most just cannot handle without retorting "It gets too hard".

Someone once summed this up rather nicely many years ago in response to issues of determinism vs probabilistic paths re observations by this phrase.

"A conclusion is where a person got tired of thinking" :-)
antialias_physorg
5 / 5 (1) Dec 04, 2013
No, because I think only those that can figure it out should know.

I'm gonna translate this as: You tried to bluff your way into sounding smart but failed miserably. Thanks for playing.

If only those that can figure it out should know then there was absolutely no reason for you to mention it at all (those who know already know, and those who don't shouldn't according to you)

Say "Hi" to Dunning-Kruger.
KBK
1 / 5 (4) Dec 05, 2013
Regardless of all the commenting wars, this algorithmic process is going to be gangbusters for generalized codebreaking.