Why The Passing of the Turing Test is a Big Deal

June 10, 2014. 

A lot of my friends don't agree with me, but I think we should be celebrating the passing of the Turing Test.

Quick summary of the Turing Test?  Can people tell the difference between a computer printing responses on a screen and a person typing?  If not then .... well I think I'll leave what the consequences are.

Three days ago was the 60th anniversary of the death of Alan Turing.  The same day, there was a big announcement: The Turing Test had been passed.  Among others, for example, see this BBC News item.

Since this happened, and it got all over the media, there's been a bit of a backlash, with many in the "Artificial Intelligentsia" commenting negatively.  (I think the lovely name for the community I am a fully signed up member of  comes from John Searle.)  I think the main points in the backlash have been:

• The narrow definition of the Turing Test that was passed is not important. Interrogators only had a 5 minute conversation and the chatbot only had to get a 30% pass rate (or fooling humans rate.)
• The winning entry cheated, or more precisely gamed the rules, by pretending to be a thirteen year old Ukrainian boy, with limited English.  
• The competition was organised by Kevin Warwick, who does not have a high level of respect in community, being viewed as a stunt-organiser. 

Taking the arguments in reverse order:


As an ad-hominem argument, the last point does not deserve response, except to express disappointment that it's even been brought up by people.  And of course to criticise it is also to criticise serious members of the AI community like Aaron Sloman and John Barnden, who took part.

There's no question the entry gamed the system.  In fact in my AI teaching at Strathclyde and St Andrews Universites for more than 15 years I've been talking about the wonderful paper "How To Pass The Turing Test By Cheating" by Jason Hutchens.  He won an earlier competition (without actually passing the Turing Test) and wrote a paper about how the tricks he used meant the Turing Test wasn't really worthwhile.  

The narrow definition of the Turing Test?  Well let's be fair here. This comes from Turing's original paper:

"I believe that in about fifty years' time it will be possible, to programme computers ... to make them play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning."

Let's take a moment to remark on Turing's remarkable prescience.  I think most people would agree that 60 years from his death (64 from the paper publication) is "about fifty years".   This man was absolutely amazing.  In fact, and I've said this before, I think Turing ranks as one of the three greatest British scientists, with Newton and Darwin.  What a guy.

So yeah, the Turing test that was passed comes from Turing's original paper. It's hard to criticise on that basis.

But no, the Turing test that was passed is definitely not important. But it's not important in the sense that it is not driving research in Artificial Intelligence. This is the sense that academics mean when they say whether something is important or not. In that sense, I agree it's not important.

But here's my point. The passing of the Turing Test is a big deal.

I mean this in a few senses.

The first one is this.  Computers are getting very good at fooling people. This is something it's good for people to know. I'd be surprised if the likes of John Barnden and Aaron Sloman were fooled, because they know what to look for in a chatbot. But people were fooled, and let's remember they knew they were participating in the Turing test. When you talk online to somebody, you might not be talking to a person even if they appear to be one.  If you're not a Ukrainian teenager, I'm not sure you should be talking to 13 year old Ukrainian boys on the internet, but still: if you think you are talking to a person, maybe you are not.

(Random aside: my surprise is that interrogators didn't wonder where the organisers had rustled up a Ukrainian teenager with good but not perfect English.)

Here is the sense in which I think the passing of the Turing Test is a very big deal indeed.

If people get the impression that Artificial Intelligence is here to stay and is playing a huge part in everyday life, they would be absolutely right.   And this is something that we in the Artificial Intelligentsia don't shout about enough.

It used to be said that Artificial Intelligence is the stuff we don't know how to get computers to do.  Because once we do know how to get them to do it, researchers in Artificial Intelligence move on to something else.   It's still true to an extent, but less so

Computer Chess?  You might remember a computer beating Garry Kasparov.  Since then computers have got much better.  Nowadays computers are dramatically better than the human world chess champion, but you don't hear about it because the champion doesn't want to get whupped by a computer so they never play.  A few years ago a mobile phone won a grandmaster level chess tournament.   That's Artificial Intelligence.  But - by the way - computers have massively improved the quality of human play, because talented teenagers can learn by playing against world class competition many times a day on their home computer.

Big data and data mining?  That's Machine Learning, probably the largest area of Artificial Intelligence.

IBM Watson beating champion quiz players in punning quizzes?  That's Artificial Intelligence.

Voice recognition, handwriting recognition?  That's Artificial Intelligence.

Computer chips do what they are meant to do?  That's Artificial Intelligence proving mathematically that they do.  When the Pentium chip first came out, there was a bug that cost Intel hundreds of millions of dollars (in mid 1990s dollars).  Now they avoid that cost through Artificial Intelligence.

Google translate?  Certainly not perfect but gives you an idea of what something written in another language is about?  That's Artificial Intelligence.

Robots who can walk? That's Artificial Intelligence.

Self driving cars on everyday roads?  That's Artificial Intelligence.  A few years ago that seemed like a ridiculous dream.  Now they're real.

Artificial Intelligence has pretty much done what it set out to do.  The above achievements are exactly the kind of things that the visionaries who founded Artificial Intelligence wanted to do.   All of the above achievements once looked like being close to science fiction: and I don't mean a long time ago, but in my memory. To give you an idea, today happens to be my 50th birthday.

 Like any other science there is much still to do.  And there are many grand challenges to overcome. The key challenge that we are nowhere close to achieving is what you might call general intelligence. All the examples above are incredibly specialised, and don't play nicely together. So I promise that the Artificial Intelligentsia is not worried yet about running out of interesting things to do with computers.

Artificial Intelligence has been just incredibly and remarkably successful.  The passing of the Turing Test three days ago is not a big deal on the scale of the achievements above.  But it does give us a chance to mark the very big deal that Artificial Intelligence has become.  If this stunt makes people more aware of this - to mark the anniversary of the tragically early death of a very great man - I think that is a good thing.

And finally, here is a punning sense in which the passing of the Turing Test is a big deal.  It's been passed, in a reasonable sense. We can stop talking about it.  The Turing Test has passed in another sense: it's over.  You can get computers to pass the Turing Test.  Big deal.