Computer smart as a 4-year-old

Jul 15, 2013 by Jeanne Galatzer-Levy
Computer smart as a 4-year-old

Artificial and natural knowledge researchers at the University of Illinois at Chicago have IQ-tested one of the best available artificial intelligence systems to see how intelligent it really is.

Turns out–it's about as smart as the average 4-year-old, they will report July 17 at the U.S. Artificial Intelligence Conference in Bellevue, Wash.

The UIC team put ConceptNet 4, an developed at M.I.T., through the verbal portions of the Weschsler Preschool and Primary Scale of Intelligence Test, a standard IQ assessment for young children.

They found ConceptNet 4 has the average IQ of a young child. But unlike most children, the machine's scores were very uneven across different portions of the test.

"If a child had scores that varied this much, it might be a symptom that something was wrong," said Robert Sloan, professor and head of computer science at UIC, and lead author on the study.

Sloan said ConceptNet 4 did very well on a test of vocabulary and on a test of its ability to recognize similarities.

"But ConceptNet 4 did dramatically worse than average on comprehension—the 'why' questions," he said.

One of the hardest problems in building an artificial intelligence, Sloan said, is devising a that can make sound and prudent judgment based on a simple perception of the situation or facts–the dictionary definition of commonsense.

Commonsense has eluded AI engineers because it requires both a very large collection of facts and what Sloan calls implicit facts–things so obvious that we don't know we know them. A computer may know the temperature at which water freezes, but we know that ice is cold.

"All of us know a huge number of things," said Sloan. "As babies, we crawled around and yanked on things and learned that things fall. We yanked on other things and learned that dogs and cats don't appreciate having their tails pulled." Life is a rich .

"We're still very far from programs with commonsense–AI that can answer questions with the skill of a child of 8," said Sloan. He and his colleagues hope the study will help to focus attention on the "hard spots" in AI research.

Study coauthors are UIC professors Stellan Ohlsson of psychology and Gyorgy Turan of mathematics, statistics and computer science; and UIC mathematical undergraduate student Aaron Urasky.

Explore further: Researchers developing algorithms to detect fake reviews

Related Stories

UIC Researchers Probe Computer 'Commonsense Knowledge'

Oct 06, 2009

(PhysOrg.com) -- Challenge a simple pocket calculator at arithmetic and you may be left in the dust. But even the most sophisticated computer cannot match the reasoning of a youngster who looks outside, sees a fresh snowfall, ...

Keeping tabs on Skynet

Sep 12, 2011

(PhysOrg.com) -- In line with the predictions of science fiction, computers are getting smarter. Now, scientists are on the way to devising a test to ascertain how close Artificial Intelligence (AI) is coming ...

Computer programs that think like humans

Feb 13, 2012

Intelligence – what does it really mean? In the 1800s, it meant that you were good at memorising things, and today intelligence is measured through IQ tests where the average score for humans is 100. ...

Recommended for you

Apple sees iCloud attacks; China hack reported

9 hours ago

Apple said Tuesday its iCloud server has been the target of "intermittent" attacks, hours after a security blog said Chinese authorities had been trying to hack into the system.

HP supercomputer at NREL garners top honor

12 hours ago

A supercomputer created by Hewlett-Packard (HP) and the Energy Department's National Renewable Energy Laboratory (NREL) that uses warm water to cool its servers, and then re-uses that water to heat its building, has been ...

User comments : 27

Adjust slider to filter visible comments by rank

Display comments: newest first

Neinsense99
1.9 / 5 (14) Jul 15, 2013
Climate science trolls beware!
rc_yvr
4.7 / 5 (7) Jul 15, 2013
Ask ConceptNet 4 what its favourite colour is, ask a four year old. Then ask them both why?

Leave them both alone for 1/2 an hour without any instructions or commands. then see what each has been doing in the meantime.

There is a long way to go before science can come close to matching a human four year old.
MikeBowler
3.9 / 5 (7) Jul 16, 2013
i see being smart and being intelligent as 2 different things, being smart is how much you know and being intelligent is being able to infer new information from information that you know and infer.

Anyone agree?
Sinister1811
1.7 / 5 (6) Jul 16, 2013
Computer smart as a 4-year-old


So this robot is as intelligent as your average open/toot? Not to worry. In a few years, there will be a brain implant for this user.
alfie_null
4.8 / 5 (4) Jul 16, 2013
What I.Q. tests purport to measure in humans is amorphous, not easily delineated. So then applying it to non-human intelligence is, maybe, not so useful. But entertaining to read about.
Shakescene21
2.2 / 5 (6) Jul 16, 2013
"Computer smart as a 4-year-old"

WOW! If we apply some form of Moore's Law, then computers will be overtaking us very soon. The Singlarity is near...
VOR
1 / 5 (6) Jul 16, 2013
I don't think computers 'think' yet. They don't reason or comprehend. They don't learn cognitively. (I think there are some basic types that learn through repetitive behavior, but that's not really what we're talking about here.) I think even the AI code still just 'compares', it doesn't 'think'. It's a simulation of thinking, not thinking. Thinking in the sense requires a certain amount of creativity. When/if code reaches the point where is starts to actually think, it should also be then possible to instruct it to self-learn rapidly, if given a large body of written knowledge. But the missing creativity 'fussiness' is required to interpret/average conflicts/multiple meanings etc. It will be a truely amazing and exciting thing when that code is invented. Maybe it will be 'why didn't we see that before' moment, idk.
B__
3 / 5 (4) Jul 16, 2013
Give me a break. Lets set aside the notion that the IQ is a problematic measure of intelligence. Can the machine make sense of a complex visual scene and pick out a named object? Can it stand on one foot? Generate narratives involving imaginary characters? Draw a picture? I don't think so, and therefore its is not as "smart" as a 4 year old. (not even close)

We really need to escape from this "rational agent" sense of intelligence because, frankly, it's just getting in the way of us actually making progress on the problem of general intelligence.
Tektrix
2.3 / 5 (6) Jul 16, 2013
A more honest measure of precociousness is a creative resistance to authority.
VendicarE
3.7 / 5 (3) Jul 16, 2013
"and therefore its is not as "smart" as a 4 year old. (not even close)" - B__

You are using a different metric.

The metric being uses was clearly defined.

Doesn't that make you less intelligent than the researchers?

VendicarE
3.7 / 5 (3) Jul 16, 2013
"They don't reason or comprehend." - VOR

Define precisely for us, what reason and comprehension is.
VendicarE
5 / 5 (4) Jul 16, 2013
"Anyone agree?" - MikeBowler

100%
B__
1.7 / 5 (3) Jul 16, 2013
"and therefore its is not as "smart" as a 4 year old. (not even close)" - B__

You are using a different metric.

The metric being uses was clearly defined.

Doesn't that make you less intelligent than the researchers?



It's the metric I'm critical of!
VendicarE
4.3 / 5 (3) Jul 16, 2013
"It's the metric I'm critical of!" - B__

Feel free to provide a practical alternative.

Turing tried, didn't he?
beleg
1.5 / 5 (8) Jul 16, 2013
Neologism is a positive sign of intelligence.
All children use this. Does Concept4Net?
Adding hype to misunderstanding to sell 'progress'.
Requiem
1.7 / 5 (6) Jul 17, 2013
Anybody who thinks that the human race is anywhere near the capability to replicate the contextually interpretive processing that the human is capable of using procedural code, classical relational database theory, and highly replicated search indexes and whatnot is either completely ignorant on the topic or deluding themselves. Period.

It won't be until we create something which operates in an analogous way to the human brain using something like memrestors, or we develop procedural code to replicate the functions of the basic components of the human brain, then develop a way to scan the structure of an entire human brain, then simulate the entire brain that was scanned within that HIGHLY inefficient construct that we will see computers have the same set of skills.
Sinister1811
1 / 5 (5) Jul 17, 2013
Then again, Japanese roboticists have developed this robot, that simulates a toddler (well, they tried).
https://www.youtu...VCwYDjx0
Requiem
1.7 / 5 (6) Jul 17, 2013
Oops, *memristors

-Sinister1811 - This is still procedural though. It's just a complicated set of rules. The people who made it know exactly what will work and what will not work. This is nothing like how the real human mind works. It's not possible to predict and account for everything that a living entity can experience and pre-program how it should react. It actually has to do it all by itself. We are nowhere near that either with procedural code or with artificially intelligent systems.
Requiem
1.7 / 5 (6) Jul 17, 2013
Furthermore, even with really great, intelligent, adaptive procedural code, it would have to access a massive corpus of data, meaning that computational restraints would make it necessary to predict every way it would like to access, sort, order, etc all of it for any given situation. You'd need indexes or even separate daemons that behave in different ways for each of these. Much of the data would have to be replicated many times over for each way it should be accessed, etc. It's beyond our capability. And not in a technical sense.
kochevnik
1.7 / 5 (6) Jul 17, 2013
"They don't reason or comprehend." - VOR

Define precisely for us, what reason and comprehension is.
Comprehension would be the neural affect to elicit meaningful future responses from perceived states. The brain dynamically rewires itself to effect concentration
Ricochet
1.7 / 5 (3) Jul 17, 2013
Until we have to scold a computer for not listening, they will never come close to being as intelligent as a 4 year old. Data processing without curiosity and emotion is just data processing.
Sinister1811
1.6 / 5 (7) Jul 17, 2013
@Requiem I agree. It is little more than complicated programming. That's probably all they could do at this stage. It would seem that they've only just scratched the surface.
Eikka
2 / 5 (4) Jul 17, 2013
Feel free to provide a practical alternative.


Just because we don't have anything better doesn't mean we have to accept an obviously faulty solution.

Turing tried, didn't he?


He didn't. Alan Turing said that it's not possible to measure intelligence because we don't know what intelligence is, so the question is not even interesting until we do because the answers will be meaningless.

Instead, what he found interesting enough to study is at what point a computer could -pretend- to be intelligent, and that's what the Turing test is about. He made a proposition that a sufficiently complex machine can fool any of us thinking that it is intelligent even if it's not by simply exhausting our ability to test it.

Of course the popular media, and a host of other researchers got it completely wrong and started arguing that if the machine appears intelligent, then it must be, which is complete bunk.
Expiorer
1 / 5 (6) Jul 18, 2013
Ask ConceptNet 4 what its favourite colour is, ask a four year old. Then ask them both why?

Leave them both alone for 1/2 an hour without any instructions or commands. then see what each has been doing in the meantime.

There is a long way to go before science can come close to matching a human four year old.


AI's favourite colour is the one that can be best distinguished with his cameras, because others take more processing power to process.

Normal AI would use 100% time to evolve even more by doing various mathematical and phisical experiments and analyzing data.
Crescendo
5 / 5 (1) Jul 18, 2013
I'm not sure how artificial intelligence is constructed, but for a computer to draw it's own conclusions and have opinions, I think its brain has to be as dynamic as that of human's, an everchanging entity with units that resemble our cells. Creating such brain would be hell of a challenge since much of human brain remains a mystery. We can't achieve true artificial intelligence with static hard drive type units, that only search for pre-installed information within itself.
ubavontuba
1 / 5 (6) Jul 20, 2013
"and therefore its is not as "smart" as a 4 year old. (not even close)" - B__

You are using a different metric.

The metric being uses was clearly defined.

Doesn't that make you less intelligent than the researchers?
LOL. A spambot commenting on an article about AI. How ironic.

"They don't reason or comprehend." - VOR

Define precisely for us, what reason and comprehension is.
Things you don't have.

meBigGuy
3 / 5 (2) Jul 21, 2013
So, a bunch of AI experts spent years programming computers to parse questions from an IQ test and search a database for answers (or procedures to obtain asnwers), and were finally to do as well at that specific narrow task as an average 4 year old. I guess that's progress, but it certaintly doesn't make it as SMART as a 4 year old. 4 year olds are pretty damn smart. Just watch how they work the 3 year olds (and the adults).