First Turing Test success marks milestone in computing history

Jun 09, 2014

(Phys.org) —An historic milestone in artificial intelligence set by Alan Turing - the father of modern computer science - has been achieved at an event organised by the University of Reading.

The 65 year-old iconic Turing Test was passed for the very first time by supercomputer Eugene Goostman during Turing Test 2014 held at the renowned Royal Society in London on Saturday.

'Eugene', a computer programme that simulates a 13 year old boy, was developed in Saint Petersburg, Russia. The development team includes Eugene's creator Vladimir Veselov, who was born in Russia and now lives in the United States, and Ukrainian born Eugene Demchenko who now lives in Russia.

The Turing Test is based on 20th century mathematician and code-breaker Turing's 1950 famous question and answer game, 'Can Machines Think?'. The experiment investigates whether people can detect if they are talking to machines or humans. The event is particularly poignant as it took place on the 60th anniversary of Turing's death, nearly six months after he was given a posthumous royal pardon.

If a computer is mistaken for a human more than 30% of the time during a series of five minute keyboard conversations it passes the test. No computer has ever achieved this, until now. Eugene managed to convince 33% of the human judges that it was human.

This historic event was organised by the University's School of Systems Engineering in partnership with RoboLaw, an EU-funded organisation examining the regulation of emerging robotic technologies.

Professor Kevin Warwick, a Visiting Professor at the University of Reading and Deputy Vice-Chancellor for Research at Coventry University, said: "In the field of Artificial Intelligence there is no more iconic and controversial milestone than the Turing Test, when a computer convinces a sufficient number of interrogators into believing that it is not a machine but rather is a human. It is fitting that such an important landmark has been reached at the Royal Society in London, the home of British Science and the scene of many great advances in human understanding over the centuries. This milestone will go down in history as one of the most exciting.

"Some will claim that the Test has already been passed. The words Turing Test have been applied to similar competitions around the world. However this event involved the most simultaneous comparison tests than ever before, was independently verified and, crucially, the conversations were unrestricted. A true Turing Test does not set the questions or topics prior to the conversations. We are therefore proud to declare that Alan Turing's Test was passed for the first time on Saturday.

"Of course the Test has implications for society today. Having a computer that can trick a human into thinking that someone, or even something, is a person we trust is a wake-up call to cybercrime. The Turing Test is a vital tool for combatting that threat. It is important to understand more fully how online, real-time communication of this type can influence an individual human in such a way that they are fooled into believing something is true...when in fact it is not."

Eugene was one of five supercomputers battling it for the Turing Test 2014 Prize. On winning the competition and achieving this historic milestone Vladimir Veselov said:

"I want to congratulate everyone who worked on Eugene Goostman. Our whole team is very excited with this result. It's a remarkable achievement for us and we hope it boosts interest in and chatbots. Special thanks to Professor Kevin Warwick and Dr Huma Shah for their effort in organising the event.

"Eugene was 'born' in 2001. Our main idea was that he can claim that he knows anything, but his age also makes it perfectly reasonable that he doesn't know everything. We spent a lot of time developing a character with a believable personality. This year we improved the 'dialog controller' which makes the conversation far more human-like when compared to programs that just answer questions. Going forward we plan to make Eugene smarter and continue working on improving what we refer to as 'conversation logic'."

Among the judges tasked with separating the human and computer participants were the actor Robert Llewellyn, who played robot Kryten in the sci-fi comedy TV series Red Dwarf, and Lord Sharkey, who led the successful campaign for Alan Turing's posthumous pardon last year.

Professor Warwick concluded: "Not long before he died on 7 June 1954 Alan Turing, himself a Fellow of the Royal Society, predicted that in time this would be passed. It is difficult to conceive that he could possibly have imagined what computers of today, and the networking that links them, would be like."

Explore further: Logic in computer science

More information: The Eugene Goostman program: www.princetonai.com/bot/bot.jsp

add to favorites email to friend print save as pdf

Related Stories

Chatbot Eugene put to Turing test wins first prize

Jun 27, 2012

(Phys.org) -- Billed as the biggest Turing test ever staged, a contest took place on June 23 in the UK, yet another event commemorating the 100th anniversary of the birth of Alan Turing. The twist is that ...

Logic in computer science

May 27, 2014

All men are mortal. Socrates is a man. Therefore, Socrates is mortal. Logical arguments like this one have been studied since antiquity. In the last few decades, however, logic research has changed considerably: ...

Mitsuku chatbot has good answers for the Loebner Prize

Sep 17, 2013

(Phys.org) —A chatbot named Mitsuku has won the Loebner Prize 2013, announced over the weekend, beating out three other contestants for the top prize of a bronze medal and $4,000. Mitsuku's creator is Steve ...

UK finally pardons computer pioneer Alan Turing

Dec 23, 2013

His code breaking prowess helped the Allies outfox the Nazis, his theories laid the foundation for the computer age, and his work on artificial intelligence still informs the debate over whether machines can think.

Recommended for you

User comments : 14

Adjust slider to filter visible comments by rank

Display comments: newest first

bodane
1 / 5 (1) Jun 09, 2014
So the time of technological solipsism is upon us.It's blatantly deceptive and could easily be used to create the crime of the century. eventually the computer generated world and physical one will merge in the a single likeness(singularity) indistinguishable from one another. some people think not to fret but it's your existential at risk and the liability could be limitless.
jscroft
not rated yet Jun 09, 2014
Is it just me, or does 30% seem like an awfully low bar?
Eikka
5 / 5 (1) Jun 09, 2014
Alan Turing predicted that the test would be passed eventually because he noted that without understanding what intelligence is, there is no way to distinguish a sufficiently complex machine from what we believe is intelligent, which is us. Since it's not possible to know intelligence without knowing intelligence, the only question you can ask is, what is complex enough?

The important point about the test is that even non-intelligent "AI" will at some point exhaust our ability to test it even if it's just a very large database of pre-recorded conversations - so do we start to call that intelligence? Why?

The popular media just puts too much of a point on the Turing test as a metric for artifical intelligence. Any time you call someone and an answering machine picks up with "hello", that's a miniature turing test, and probably more than half of the people will be fooled for at least a few seconds, so why is it significant if we are fooled for five minutes, an hour, or indefinitely?
jscroft
not rated yet Jun 09, 2014
... so why is it significant if we are fooled for five minutes, an hour, or indefinitely?


I don't think it's ridiculous to assert that a difference in magnitude can ultimately become a difference in KIND.

Think of a Turing test as integrative: the longer it runs, the lower the uncertainty in the final determination (at least to some asymptotic limit). So I guess any reasonably effective Turing test would have to run "long enough", probably defined as the point beyond which the judges' scores fail to shift by a significant amount.

I'm still balking at calling a 30% success rate a "pass". Seems like that threshold should be 50% at least!
SoylentGrin
5 / 5 (1) Jun 09, 2014
The singularity is not upon us yet. An AI that is able to hold a conversation based on knowledge gathered for it would make an interesting virtual assistant, but it would still be at (at most) or more likely far below a human's capability to generate knowledge.
Once we have computers that are able to gather and assess data, test and formulate hypotheses, and build autonomously on that knowledge... and do it faster than humans can... then we have something. ChatBots are still relegated to human levels of knowledge.
antialias_physorg
not rated yet Jun 09, 2014
Is it just me, or does 30% seem like an awfully low bar?

It's enough to get elected in most countries (if you consider the number of people who don't vote)

That this program can fool that many people is pretty amazing. Intelligence is just a quantitative effect - not a qualitative one (as many animals can behave intelligently). So just like with speech recognition and automated translation this will only get better to the point wher it doesn't really matter whether there is any true cognition going on in the background or not (for the overwhelming majority of cases).
jscroft
not rated yet Jun 09, 2014
Sure it's impressive. I just question whether it's impressive enough to justify the headline.
kelman66
not rated yet Jun 09, 2014
30% may be a low threshold but it still took nearly 65 years since Turing postulated this.

Regardless, any recognition of Turing is well deserved as he was one of the greatest minds of the 20th century. Perhaps no other individual had a greater affect on the outcome of World War 2. Ten years after the war, and before the secret work on Enigma code breaking and Turing's incredible genius was known, he was charged with Homosexuality and chemically castrated. He committed suicide a short time later.
zaxxon451
not rated yet Jun 10, 2014
Sorry, I really don't think Turing would be impressed. Placing restrictions like 13y/o ESL Ukrainian is just a crutch to muddy the waters. Any programmer could code a chatbot that could pass the test with enough limitations placed on the characteristics of the "human". The poor language skills issue is particularly underhanded in my opinion.

WIRED: Where are you from?
Goostman: A big Ukrainian city called Odessa on the shores of the Black Sea

WIRED: Oh, I'm from the Ukraine. Have you ever been there?
Goostman: ukraine? I've never there. But I do suspect that these crappy robots from the Great Robots Cabal will try to defeat this nice place too.

http://www.wired....so-fast/
Eikka
not rated yet Jun 10, 2014
I don't think it's ridiculous to assert that a difference in magnitude can ultimately become a difference in KIND.


True, but you have to explain why adding more grains turns it into a heap, and in this case that involves proving that intelligence is ultimately just the same thing as a sufficiently complex answering machine - which is intuitively implausible because we certainly don't percieve ourselves as just walking databanks of stored conversations that cut and paste statistically appropriate phrases as answers to specific input.

Think of a Turing test as integrative: the longer it runs, the lower the uncertainty in the final determination


Just asserting that it is because we can't tell the difference anymore is saying that cats are dogs because we forgot to put on our spectacles.

Eikka
not rated yet Jun 10, 2014
Intelligence is just a quantitative effect - not a qualitative one (as many animals can behave intelligently).


Qualitively, animals have brains and not digital processors running deterministic algorithms.

So what proof does animal intelligence have on machine intelligence?

this will only get better to the point wher it doesn't really matter whether there is any true cognition going on in the background or not


The point where it matters is whether we can say that people are truly intelligent any longer, or deserving of human rights, because we can't make the distinction between a sufficiently complex machine and a living person.
Eikka
not rated yet Jun 10, 2014
In one dystopic view, people below certain IQ and/or educational level could be treated as machines with no moral agency and no rights, because they aren't "complex enough" to qualify as intelligent and by proxy, truly conscious.

Simply because treating intelligence as simply a sufficiently complex algorithm would necessarily create an arbitrary distinction between what is and what isn't, while denying that the difference should make any difference would create the problem of having to treat televisions and thermostats as people.

That of course is an appeal to consequences and cannot truly decide the fact of the matter, but it is an issue to be aware of.
jscroft
not rated yet Jun 11, 2014
I wonder if Scott Nudds could pass a Turing test.
baudrunner
not rated yet Jun 14, 2014
The researchers who conducted the test and passed Eugene proved that they were human, because humans are prone to error. Go to Eugene's chatbot and see if you can't fool Eugene within three questions. (link is found on this page: https://www.yahoo...919.html ) Just because he answers every question does not mean that he can fool anyone. He does not respond like an ordinary 13 year old. He fails, in my book.