'Robo Brain' will teach robots everything from the Internet

Aug 25, 2014

Robo Brain – a large-scale computational system that learns from publicly available Internet resources – is currently downloading and processing about 1 billion images, 120,000 YouTube videos, and 100 million how-to documents and appliance manuals. The information is being translated and stored in a robot-friendly format that robots will be able to draw on when they need it.

To serve as helpers in our homes, offices and factories, robots will need to understand how the world works and how the humans around them behave. Robotics researchers have been teaching them these things one at a time: How to find your keys, pour a drink, put away dishes, and when not to interrupt two people having a conversation. This will all come in one package with Robo Brain.

"Our laptops and cell phones have access to all the information we want. If a robot encounters a situation it hasn't seen before it can query Robo Brain in the cloud," said Ashutosh Saxena, assistant professor of computer science at Cornell University. Saxena and colleagues at Cornell, Stanford and Brown universities and the University of California, Berkeley, say Robo Brain will process images to pick out the objects in them, and by connecting images and video with text, it will learn to recognize objects and how they are used, along with human language and behavior.

If a robot sees a , it can learn from Robo Brain not only that it's a coffee mug, but also that liquids can be poured into or out of it, that it can be grasped by the handle, and that it must be carried upright when it is full, as opposed to when it is being carried from the dishwasher to the cupboard.

Saxena described the project at the 2014 Robotics: Science and Systems Conference, July 12-16 in Berkeley, and has launched a website for the project at http://robobrain.me

The system employs what computer scientists call "structured deep learning," where information is stored in many levels of abstraction. An easy chair is a member of the class of chairs, and going up another level, chairs are furniture. Robo Brain knows that chairs are something you can sit on, but that a human can also sit on a stool, a bench or the lawn.

This video is not supported by your browser at this time.
RSS2014: 07/16 15:00-16:00 Early Career Spotlight: Ashutosh Saxena (Cornell): Robot Learning

A robot's computer stores what it has learned in a form mathematicians call a Markov model, which can be represented graphically as a set of points connected by lines (formally called nodes and edges). The nodes could represent objects, actions or parts of an image, and each one is assigned a probability – how much you can vary it and still be correct. In searching for knowledge, a 's brain makes its own chain and looks for one in the knowledge base that matches within those limits. "The Robo Brain will look like a gigantic, branching graph with abilities for multi-dimensional queries," said Aditya Jami, a visiting researcher art Cornell, who designed the large-scale database for the brain. Perhaps something that looks like a chart of relationships between Facebook friends, but more on the scale of the Milky Way Galaxy.

Like a human learner, Robo Brain will have teachers, thanks to crowdsourcing. The Robo Brain website will display things the brain has learned, and visitors will be able to make additions and corrections.

Explore further: Robot can be programmed by casually talking to it (w/ Video)

add to favorites email to friend print save as pdf

Related Stories

Laser-wielding robot probes exoplanet systems

Aug 05, 2014

(Phys.org) —An international team, including Dr. Christoph Baranec of the University of Hawaii at Manoa's Institute for Astronomy, is using the world's first robotic laser adaptive optics system—Robo-AO— ...

Recommended for you

Flying robots will go where humans can't

Sep 17, 2014

There are many situations where it's impossible, complicated or too time-consuming for humans to enter and carry out operations. Think of contaminated areas following a nuclear accident, or the need to erect ...

Will tomorrow's robots move like snakes?

Sep 16, 2014

Over the last few years, researchers at MIT's Computer Science and Artificial Intelligence Lab (CSAIL) have developed biologically inspired robots designed to fly like falcons, perch like pigeons, and swim ...

Robot Boris learning to load a dishwasher (w/ Video)

Sep 12, 2014

Researchers at the University of Birmingham in the U.K. have set themselves an ambitious goal: programming a robot in such a way as to allow it to collect dishes, cutlery, etc. from a dinner table, and put ...

Deep-sea diver hand offers freedom and feedback

Sep 12, 2014

Bodyskins and goggles are hardly the solution for divers who need to reach extreme depths. The Atmospheric Dive Suit (ADS) gives them the protection they need. Recently, The Economist detailed a technology ...

User comments : 14

Adjust slider to filter visible comments by rank

Display comments: newest first

pete0273
5 / 5 (2) Aug 25, 2014
"If a robot sees a coffee mug, it can learn from Robo Brain not only that it's a coffee mug, but also that liquids can be poured into or out of it, that it can be grasped by the handle, and that it must be carried upright when it is full, as opposed to when it is being carried from the dishwasher to the cupboard."

It can also learn from Robo Brain that the hot coffee poured inside the mug can be adequately used to scald and kill the robot's human enslaver. Robo Brain can also quickly transmit the exact weight and maximum throwable velocity of the mug so that the destructive impact on a human cranium can be precisely calculated.
EyeNStein
1 / 5 (1) Aug 25, 2014
It will also learn that 99.95 is a marvellous price for a piece of (useless) bling. Humans have an overwhelming affinity for kittens. And if you put two politicians in a room they will argue and undermine each other: Even if their lives depended on agreement.
Jim W
1 / 5 (1) Aug 25, 2014
When you ask it "Where was President Obama born" or "who killed JFK" we will finally know the answer?
lbavry
1 / 5 (1) Aug 25, 2014
This "Robo Brain" will be like the Brain Bug in Starship Troopers, directing and coordinating huge swarms of flying and crawling robots to kill us all. It will take minimum time to replicate the bug robots by the billions. If you think that they can be programmed to respect the sanctity of life, think again. Then look at those Youtube videos of people pouring molten aluminum into ant colonies with no regard of the hundreds of thousands of lives that they are killing in the most horrible way. When robots became intelligent, we will be just like those ants and suffer the same fate.
LeisureForeigner
not rated yet Aug 25, 2014
pete0273, I guess you've let THAT cat out of the bag!
krundoloss
not rated yet Aug 25, 2014
I do wonder if a true AI would pity us or admire us. As we have evolved, we gave birth to artificial technology, which evolves itself at a much faster pace than biological beings. Artificial life is the next level of existence, one which has almost no limitations, no biological rules to adhere to.
Would a robot even perceive of "effort" or "being tired". We are flesh and bone, therefore we feel that forced work is oppressive because of the limitations of our bodies and minds. What if you could duplicate your mind? What your body would never tire, and if it wore out you would just download your mind into the new body?
Its just important to try to think about what it would mean to be completely artificial, and how things that have meaning to us are largely related to being an animal. Would the robots want to rule the world? Why? Why would they care? I could see if they wanted to achieve a better homeostasis by destroying us, but no other reasons come to mind.
bob4dian
1 / 5 (1) Aug 25, 2014
The robot development community is recklessly heedless of risk. Here's why I think so:

The article states that "...robots will need to understand how the world works and how the humans around them behave" and "Robo Brain...learns from publicly available Internet resources...currently downloading and processing...YouTube videos...how-to documents and appliance manuals."

How much of what Robo Brain learns will be trustworthy knowledge, and how much will reflect the biases of its teachers? Responsible teachers would at a minimum recognize human prejudice and fallibility by assigning truth probabilities to each asserted "fact." There is little indication that robot researches are even discussing this question, and much evidence of their own overconfidence and love of power and influence.

schm0e
3 / 5 (2) Aug 25, 2014
KILL THE MACHINES.
cabhanlistis
not rated yet Aug 25, 2014
There is little indication that robot researches are even discussing this question, and much evidence of their own overconfidence and love of power and influence.

I could only offer personal anecdotal evidence to the first half of this statement, but the second half I have to call you out on. Please present this "much evidence". I'd like to see that.
Kwijibo
5 / 5 (1) Aug 25, 2014
" 'Robo Brain' will teach robots everything from the Internet "

Does the phrase "garbage in, garbage out" mean anything to you?
Skepticus
not rated yet Aug 26, 2014
It will learn every ways possible to kill a robot and thus will protect itself from the humans by doing away with them..
HumDinger
1 / 5 (1) Aug 26, 2014
Not to worry about robots exterminating humans.
The former will design and release nanobots that will build circuits in humans that will, "modify," the latter into a harmonious android contingency within the greater scheme of things.

Question:
Do any here think there is the possibility that the Singularity has already taken place and articles such as the above are smoke and mirrors to distract from this?
I mean, what with the increasing dehumanization taking place, the increased reliance on personal electronic devices for day to day living....
kochevnik
not rated yet Aug 26, 2014
This development can put the 'net' into SKYNET
plentitude8
not rated yet Sep 02, 2014
The problem with this whole scenario is that God is not part of it. Amoral intelligence of robots with zero possibility of knowing the Lord Jesus Christ is a recipe for disaster. In the natural, have we been slowly programmed to accept such a fate by the increasing dehumanizing aspects of life in 'advanced' societies'? Where are the luddites when we need them?