Why The Passing of the Turing Test is a Big Deal

June 10, 2014. 

A lot of my friends don't agree with me, but I think we should be celebrating the passing of the Turing Test.

Quick summary of the Turing Test?  Can people tell the difference between a computer printing responses on a screen and a person typing?  If not then .... well I think I'll leave what the consequences are.

Three days ago was the 60th anniversary of the death of Alan Turing.  The same day, there was a big announcement: The Turing Test had been passed.  Among others, for example, see this BBC News item.

Since this happened, and it got all over the media, there's been a bit of a backlash, with many in the "Artificial Intelligentsia" commenting negatively.  (I think the lovely name for the community I am a fully signed up member of  comes from John Searle.)  I think the main points in the backlash have been:

• The narrow definition of the Turing Test that was passed is not important. Interrogators only had a 5 minute conversation and the chatbot only had to get a 30% pass rate (or fooling humans rate.)
• The winning entry cheated, or more precisely gamed the rules, by pretending to be a thirteen year old Ukrainian boy, with limited English.  
• The competition was organised by Kevin Warwick, who does not have a high level of respect in community, being viewed as a stunt-organiser. 

Taking the arguments in reverse order:


As an ad-hominem argument, the last point does not deserve response, except to express disappointment that it's even been brought up by people.  And of course to criticise it is also to criticise serious members of the AI community like Aaron Sloman and John Barnden, who took part.

There's no question the entry gamed the system.  In fact in my AI teaching at Strathclyde and St Andrews Universites for more than 15 years I've been talking about the wonderful paper "How To Pass The Turing Test By Cheating" by Jason Hutchens.  He won an earlier competition (without actually passing the Turing Test) and wrote a paper about how the tricks he used meant the Turing Test wasn't really worthwhile.  

The narrow definition of the Turing Test?  Well let's be fair here. This comes from Turing's original paper:

"I believe that in about fifty years' time it will be possible, to programme computers ... to make them play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning."

Let's take a moment to remark on Turing's remarkable prescience.  I think most people would agree that 60 years from his death (64 from the paper publication) is "about fifty years".   This man was absolutely amazing.  In fact, and I've said this before, I think Turing ranks as one of the three greatest British scientists, with Newton and Darwin.  What a guy.

So yeah, the Turing test that was passed comes from Turing's original paper. It's hard to criticise on that basis.

But no, the Turing test that was passed is definitely not important. But it's not important in the sense that it is not driving research in Artificial Intelligence. This is the sense that academics mean when they say whether something is important or not. In that sense, I agree it's not important.

But here's my point. The passing of the Turing Test is a big deal.

I mean this in a few senses.

The first one is this.  Computers are getting very good at fooling people. This is something it's good for people to know. I'd be surprised if the likes of John Barnden and Aaron Sloman were fooled, because they know what to look for in a chatbot. But people were fooled, and let's remember they knew they were participating in the Turing test. When you talk online to somebody, you might not be talking to a person even if they appear to be one.  If you're not a Ukrainian teenager, I'm not sure you should be talking to 13 year old Ukrainian boys on the internet, but still: if you think you are talking to a person, maybe you are not.

(Random aside: my surprise is that interrogators didn't wonder where the organisers had rustled up a Ukrainian teenager with good but not perfect English.)

Here is the sense in which I think the passing of the Turing Test is a very big deal indeed.

If people get the impression that Artificial Intelligence is here to stay and is playing a huge part in everyday life, they would be absolutely right.   And this is something that we in the Artificial Intelligentsia don't shout about enough.

It used to be said that Artificial Intelligence is the stuff we don't know how to get computers to do.  Because once we do know how to get them to do it, researchers in Artificial Intelligence move on to something else.   It's still true to an extent, but less so

Computer Chess?  You might remember a computer beating Garry Kasparov.  Since then computers have got much better.  Nowadays computers are dramatically better than the human world chess champion, but you don't hear about it because the champion doesn't want to get whupped by a computer so they never play.  A few years ago a mobile phone won a grandmaster level chess tournament.   That's Artificial Intelligence.  But - by the way - computers have massively improved the quality of human play, because talented teenagers can learn by playing against world class competition many times a day on their home computer.

Big data and data mining?  That's Machine Learning, probably the largest area of Artificial Intelligence.

IBM Watson beating champion quiz players in punning quizzes?  That's Artificial Intelligence.

Voice recognition, handwriting recognition?  That's Artificial Intelligence.

Computer chips do what they are meant to do?  That's Artificial Intelligence proving mathematically that they do.  When the Pentium chip first came out, there was a bug that cost Intel hundreds of millions of dollars (in mid 1990s dollars).  Now they avoid that cost through Artificial Intelligence.

Google translate?  Certainly not perfect but gives you an idea of what something written in another language is about?  That's Artificial Intelligence.

Robots who can walk? That's Artificial Intelligence.

Self driving cars on everyday roads?  That's Artificial Intelligence.  A few years ago that seemed like a ridiculous dream.  Now they're real.

Artificial Intelligence has pretty much done what it set out to do.  The above achievements are exactly the kind of things that the visionaries who founded Artificial Intelligence wanted to do.   All of the above achievements once looked like being close to science fiction: and I don't mean a long time ago, but in my memory. To give you an idea, today happens to be my 50th birthday.

 Like any other science there is much still to do.  And there are many grand challenges to overcome. The key challenge that we are nowhere close to achieving is what you might call general intelligence. All the examples above are incredibly specialised, and don't play nicely together. So I promise that the Artificial Intelligentsia is not worried yet about running out of interesting things to do with computers.

Artificial Intelligence has been just incredibly and remarkably successful.  The passing of the Turing Test three days ago is not a big deal on the scale of the achievements above.  But it does give us a chance to mark the very big deal that Artificial Intelligence has become.  If this stunt makes people more aware of this - to mark the anniversary of the tragically early death of a very great man - I think that is a good thing.

And finally, here is a punning sense in which the passing of the Turing Test is a big deal.  It's been passed, in a reasonable sense. We can stop talking about it.  The Turing Test has passed in another sense: it's over.  You can get computers to pass the Turing Test.  Big deal.

Yes For Independence, Because Eton

Last night I attended a debate about Scottish Independence at the University of St Andrews.  It was very good and well tempered on both sides.  

One of the arguments from the Yes side was that the UK Cabinet is dominated by Etonians. Obviously this is a major problem with Westminster politics, since one only needs to look at the education of the last few prime ministers.   I've examined the evidence and present it to you now, and you can see the point is completely definitive.  

• Eton.  There you are.  David Cameron went to Eton. You see!  The point is proved. 

You might want to look away now, because is there any necessity to look back further?   You want to?  Ok.

• Kirkcaldy High School.  Well the exception that proves the rule, I suppose. 

• Fettes College.  Well that is an Edinburgh school, but it has been called "Eton of the North". So maybe we can count it.  So yep, let's count it as showing the dominance of Eton, or at least posh public schools. 

Really, look away.  Stop reading.  The point is made!  

• Rutlish Grammar School.  Perhaps not a surprise that John Major didn't go to a major public school.  Or even a public school.  But again, he's the Conservative exception to the rule.

• Kesteven and Grantham Girls' School.  Maggie obviously couldn't go to Eton, as a woman. As the Conservative hate figure of recent times, she must have gone to a posh public school. Though oddly the school's history seems to suggest it was a grammar school.
  

Really, there is no need to carry on reading.  A couple of Tory Prime Ministers didn't go to Eton, and one of them was a woman so she couldn't.  

• Portsmouth Northern Secondary School. Sunny Jim Callaghan.  Labour.

• Royds Hall Grammar School, Huddersfield.  Harold Wilson may have been photographed on the steps of Downing St as a boy, but he went to a grammar school. 

• Chatham House Grammar School, Ramsgate.  As did his predecessor as Prime Minister, Ted Heath.  That good old Labour, no ... umm, ok another Tory grammar school boy.

• Eton.  There you are!  The point is confirmed.  Of course it's a bit embarrassing that this was a Scottish Prime Minister, Alec Douglas-Home, but perhaps we could pass a law banning First Ministers from having attended Eton.  

From now on backwards it's really easy to see the point, because it goes Eton (Macmillan), Eton (Eden), Harrow (Churchill), and Haileybury (Attlee).   More public schoolboys back to the Second World War.  So there you have it, from 1940 to 1964 and from 2010 to 2014, the UK has been governed by English public school pupils. 

Pay no attention to the period between 1964 and 2010, because it's irrelevant.  The fact that not a single Prime Minister of the United Kingdom in 46 years attended an English public school is totally irrelevant.

So if you don't like Eton, vote Yes for Independence. 

Author's note.  Like my previous piece, for avoidance of doubt I will state that this piece is not serious.  The point is to show that dominance of UK politics by Etonians, or even public school pupils, is not true. 

For avoidance of any more doubt, I don't think the educational makeup of the current cabinet is reason to vote against them.  The reason to vote against them is because you don't like their policies. I think that is a very good reason to vote against them: whether or not that is a good reason to vote for independence is another question.