Robots learning from experience (w/ Video)

August 24, 2010, ICT Results
Robots learning from experience

Software that enables robots to move objects about a room, building up ever-more knowledge about their environment, is an important step forward in artificial intelligence.

Some objects can be moved, while others cannot. Balls can be placed on top of boxes, but boxes cannot be stably stacked on top of balls. A typical one-year-old child can discover this kind of information about its environment very quickly. But it is a massive challenge for a robot - a machine - to learn concepts such as ‘movability’ and ‘stability’, according to Björn Kahl, a researcher at the Bonn-Rhein-Sieg University and a member of the Xpero robotics research project team.

The aim of the Xpero project was to develop a cognitive system for a robot that would enable it to explore the world around it and learn through physical experimentation.

Logically testing hypotheses

The first step was to create an that enabled the robot to discover its environment from data it received from its sensors.  The Xpero researchers installed some very basic predefined ‘knowledge’ into the robot. That knowledge is based on logic. The robot believes that things are either true or false - there are no ‘maybes’.  The robot uses the data from its sensors as it moves about to test that knowledge. When the robot finds that an expectation is false it starts to experiment to find out why it is false and to correct its hypotheses.

Picking out the important factors in the massive and continuous flow of data from the robot’s created one challenge for the EU-funded Xpero project team. Finding a way for a logic-based system to deal with the concept of time was a second challenge.

Initially the robot has no useful vision of the probable future, but with each observation it learns better hypotheses that it can use to predict the effects of its actions. If an experiment showed that one of its hypotheses was false, then there were literally an infinite number of possibilities of what the correct solution might be. The team had to find ways to short-circuit the process to stop the robot spending an infinite amount of time testing each possibility.

Part of the Xpero team’s solution was to ignore some of the flow of data coming in every millisecond and instead to get the robot to compare snapshots of the situation after a few seconds. When an expectation proved false they also cut down the possible number of solutions by getting the robot to build a new hypothesis that kept the logic connectors from its old hypothesis, simply changing the variables. That drastically reduced the number of possible solutions.

Building a store of knowledge

An important development from Xpero is the robot’s ability to build its knowledge base. “It makes no distinction between previous knowledge and learnt knowledge,” explains Kahl. “That it can re-use is very important. Without that there would be no incremental learning.”

In award-winning demonstrations, robots with the Xpero cognitive system on board have moved about, pushed and placed objects, learning all the time about their environment. In an exciting recent development the robot has started to use objects as tools. It has used one object to move or manipulate another object that it cannot reach directly.

The robot notices new objects in his world. Some are boxes and some are balls. They are smaller then the other ones, so he tries to grab them. Achieving this, he goes even further and starts placing objects on top of each other. Experimenting this way, he learns that structures built by placing objects on boxes are stable, whereas the ones where he places objects on balls are not.

The objects in the world interest the robot. He would like to see if he can manipulate them, so he attempts to push around each individual object. Some stand firm, others he manages to move. Using the knowledge from these experiments, he induces the concept of movability, by describing the two possible outcomes of his pushing action.
While exploring robots makes great theatre, the most exciting developments to come out of Xpero are what the team learnt about the process of learning itself, says Kahl. “We gained a lot of insight into what the challenges in learning are and how machine-learning really works. Just getting the robot to figure out that something is not right required major insights from a research point of view.”

They are planning a new project that will run one or two robots for a much longer time - perhaps months - to see how they advance.

The Xpero project lays the first cornerstones for a technology that has the potential to become a key technology for the next generation of so-called service robots, which clean our houses and mow our lawns - replacing the rather dumb, pre-programmed devices on the market today. A robotics manufacturer is already planning to use parts of the Xpero platform in the edutainment market.

“But while Xpero advances machine learning, it is still far short of the capabilities of a baby,” says Kahl. “Of course, the can now learn the concept of movability. But it does not understand in the human sense what movability means.”

Explore further: 'What can I, Robot, do with that?'

More information: Xpero project -

Related Stories

'What can I, Robot, do with that?'

April 21, 2008

A new approach to robotics and artificial intelligence (AI) could lead to a revolution in the field by shifting the focus from what a thing is to how it can be used.

Recommended for you

Cryptocurrency rivals snap at Bitcoin's heels

January 14, 2018

Bitcoin may be the most famous cryptocurrency but, despite a dizzying rise, it's not the most lucrative one and far from alone in a universe that counts 1,400 rivals, and counting.


Adjust slider to filter visible comments by rank

Display comments: newest first

4.8 / 5 (4) Aug 24, 2010
The pre-programmed "thought process" was a little much. Other than that, it was interesting. I just remind myself that just a few years ago, bending down and picking up a box would have been the entire article.
3 / 5 (1) Aug 24, 2010
The monologue is a little bit too much...
2 / 5 (1) Aug 24, 2010
Anyone else notice that the article refers to the robot as "He"?
5 / 5 (2) Aug 24, 2010
Perhaps the robot has a removable male appendage.
4 / 5 (1) Aug 25, 2010
Quite cute. Perhaps if they injected a bit of fuzzy logic (as in the Bayesian logic chip recently in the news), instead of just relying on Boolean true/false logic, it might be able to improve its world modeling & learning behaviour.

I just wish there was a more coordinated and on-going effort in this type of research rather then continually seeing individual teams doing some 'cool' snippets of research which is quickly forgotten before another team tries something else from scratch.

I remember reading ages ago about Shakey the robot (SRI International) doing something similar (exploring a room) some 44 years ago!
2 / 5 (2) Aug 25, 2010
they call this "an important step forward in artificial intelligence"? very disappointing. to me it looks like regular pre-programmed stuff.
5 / 5 (1) Aug 25, 2010
they call this "an important step forward in artificial intelligence"? very disappointing. to me it looks like regular pre-programmed stuff.

How can you tell the nature of the software just by looking at a robot video?
4 / 5 (1) Aug 25, 2010
Well, at least for me, it is impressive.
3.4 / 5 (5) Aug 25, 2010
How can you tell the nature of the software just by looking at a robot video?

well, it's very easy to deduce from the video. especially for a software engeneer. its behaviour is split into cycles consisting of obvious stages, like looking around, recognizing objects and matching them against database, moving into position that allows picking up an object, picking it up and performing some predefined action against some other object, analyzing the result basing on predefined algorithm and saving it into database. they call it "learning". i'd call it "performing automated tests". it has nothing to do with a "real" AI.
2 / 5 (2) Aug 25, 2010
I Agree. The reason children don't spend an infinite time on a task is because they lose interest. There's a mental reward mechanism in everything we do. Trying new things that interest us causes a mental reward (good feeling). Eventually that wains, and we lose interest. The good feeling is triggered by chemical reactions that make us physically feel good. It's millions of years of complex evolution that brought us a balance of these interactions and feelings. Trying to mimic that is not AI. Reproducing it is AI.
4.5 / 5 (2) Aug 25, 2010
hagureinu, you make it sound like 'looking around, recognizing objects and matching them against a database [memory], moving into position that allows picking up an object' aren't functions that a person performs, even if they make no conscious thought in doing it (having been learned).

it has nothing to do with a "real" AI.

I'd like to know what 'real' AI is. Yes, it's obvious that the robot has some preset kinematic movements, like kneeling down to move a box, but the smarts isn't so much in locomotion but in learned rules and inferences that it makes about its environment using its senses. And that, you can't really see in the video, which is why the somewhat cheesy voice-over is used (to explain what is happening behind the scenes, the AI bit).
5 / 5 (3) Aug 25, 2010
They should face off a bunch of these robots, so they can have "robot stacking wars."
3 / 5 (1) Aug 26, 2010
I'd like to know what 'real' AI is.

in essence, intelligence is is something that allows organism to predict future and choose actions that lead to successfull adaptation in the environment. the key point is abitilty to build and learn models of things that happen in the world around. e.g. if that robot was able to grasp the concept of 'stable' on the fly without preliminary hard-coding it into its algorithm set, then it defenitely would be intelligent. but at the moment it is not even close.
2.7 / 5 (3) Aug 26, 2010
If this robot is what we can do today in the year 2010, then imagine what will be possible in just 100 years. Better yet, imagine what an alien civilization, of which there are thousands in the Universe, can create if sufficiently advanced, which, according to recent mathematical calculations, should be about one-tenth of all civilizations out there, namely HUNDREDS!
3 / 5 (2) Aug 27, 2010
It's obvious. Boxes with light blue tops are immovable! Everyone knows that.

5 / 5 (1) Aug 30, 2010
This is not even remotely close to intelligence.
Intelligence is creative and intuitive.
This is simply a complex set of decision gates pretending to simulate intelligence.
Will the computer industry ever become honest or is that an impossibility?

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.