Tuesday, 10 June 2014

Why The Passing of the Turing Test is a Big Deal

June 10, 2014. 

A lot of my friends don't agree with me, but I think we should be celebrating the passing of the Turing Test.

Quick summary of the Turing Test?  Can people tell the difference between a computer printing responses on a screen and a person typing?  If not then .... well I think I'll leave what the consequences are.

Three days ago was the 60th anniversary of the death of Alan Turing.  The same day, there was a big announcement: The Turing Test had been passed.  Among others, for example, see this BBC News item.

Since this happened, and it got all over the media, there's been a bit of a backlash, with many in the "Artificial Intelligentsia" commenting negatively.  (I think the lovely name for the community I am a fully signed up member of  comes from John Searle.)  I think the main points in the backlash have been:

  • The narrow definition of the Turing Test that was passed is not important. Interrogators only had a 5 minute conversation and the chatbot only had to get a 30% pass rate (or fooling humans rate.)
  • The winning entry cheated, or more precisely gamed the rules, by pretending to be a thirteen year old Ukrainian boy, with limited English.  
  • The competition was organised by Kevin Warwick, who does not have a high level of respect in community, being viewed as a stunt-organiser. 
Taking the arguments in reverse order:

As an ad-hominem argument, the last point does not deserve response, except to express disappointment that it's even been brought up by people.  And of course to criticise it is also to criticise serious members of the AI community like Aaron Sloman and John Barnden, who took part.

There's no question the entry gamed the system.  In fact in my AI teaching at Strathclyde and St Andrews Universites for more than 15 years I've been talking about the wonderful paper "How To Pass The Turing Test By Cheating" by Jason Hutchens.  He won an earlier competition (without actually passing the Turing Test) and wrote a paper about how the tricks he used meant the Turing Test wasn't really worthwhile.  

The narrow definition of the Turing Test?  Well let's be fair here. This comes from Turing's original paper:
"I believe that in about fifty years' time it will be possible, to programme computers ... to make them play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning."
Let's take a moment to remark on Turing's remarkable prescience.  I think most people would agree that 60 years from his death (64 from the paper publication) is "about fifty years".   This man was absolutely amazing.  In fact, and I've said this before, I think Turing ranks as one of the three greatest British scientists, with Newton and Darwin.  What a guy.

So yeah, the Turing test that was passed comes from Turing's original paper. It's hard to criticise on that basis.

But no, the Turing test that was passed is definitely not important. But it's not important in the sense that it is not driving research in Artificial Intelligence. This is the sense that academics mean when they say whether something is important or not. In that sense, I agree it's not important.

But here's my point. The passing of the Turing Test is a big deal.

I mean this in a few senses.

The first one is this.  Computers are getting very good at fooling people. This is something it's good for people to know. I'd be surprised if the likes of John Barnden and Aaron Sloman were fooled, because they know what to look for in a chatbot. But people were fooled, and let's remember they knew they were participating in the Turing test. When you talk online to somebody, you might not be talking to a person even if they appear to be one.  If you're not a Ukrainian teenager, I'm not sure you should be talking to 13 year old Ukrainian boys on the internet, but still: if you think you are talking to a person, maybe you are not.

(Random aside: my surprise is that interrogators didn't wonder where the organisers had rustled up a Ukrainian teenager with good but not perfect English.)

Here is the sense in which I think the passing of the Turing Test is a very big deal indeed.

If people get the impression that Artificial Intelligence is here to stay and is playing a huge part in everyday life, they would be absolutely right.   And this is something that we in the Artificial Intelligentsia don't shout about enough.

It used to be said that Artificial Intelligence is the stuff we don't know how to get computers to do.  Because once we do know how to get them to do it, researchers in Artificial Intelligence move on to something else.   It's still true to an extent, but less so

Computer Chess?  You might remember a computer beating Garry Kasparov.  Since then computers have got much better.  Nowadays computers are dramatically better than the human world chess champion, but you don't hear about it because the champion doesn't want to get whupped by a computer so they never play.  A few years ago a mobile phone won a grandmaster level chess tournament.   That's Artificial Intelligence.  But - by the way - computers have massively improved the quality of human play, because talented teenagers can learn by playing against world class competition many times a day on their home computer.

Big data and data mining?  That's Machine Learning, probably the largest area of Artificial Intelligence.

IBM Watson beating champion quiz players in punning quizzes?  That's Artificial Intelligence.

Voice recognition, handwriting recognition?  That's Artificial Intelligence.

Computer chips do what they are meant to do?  That's Artificial Intelligence proving mathematically that they do.  When the Pentium chip first came out, there was a bug that cost Intel hundreds of millions of dollars (in mid 1990s dollars).  Now they avoid that cost through Artificial Intelligence.

Google translate?  Certainly not perfect but gives you an idea of what something written in another language is about?  That's Artificial Intelligence.

Robots who can walk? That's Artificial Intelligence.

Self driving cars on everyday roads?  That's Artificial Intelligence.  A few years ago that seemed like a ridiculous dream.  Now they're real.

Artificial Intelligence has pretty much done what it set out to do.  The above achievements are exactly the kind of things that the visionaries who founded Artificial Intelligence wanted to do.   All of the above achievements once looked like being close to science fiction: and I don't mean a long time ago, but in my memory. To give you an idea, today happens to be my 50th birthday.

 Like any other science there is much still to do.  And there are many grand challenges to overcome. The key challenge that we are nowhere close to achieving is what you might call general intelligence. All the examples above are incredibly specialised, and don't play nicely together. So I promise that the Artificial Intelligentsia is not worried yet about running out of interesting things to do with computers.

Artificial Intelligence has been just incredibly and remarkably successful.  The passing of the Turing Test three days ago is not a big deal on the scale of the achievements above.  But it does give us a chance to mark the very big deal that Artificial Intelligence has become.  If this stunt makes people more aware of this - to mark the anniversary of the tragically early death of a very great man - I think that is a good thing.

And finally, here is a punning sense in which the passing of the Turing Test is a big deal.  It's been passed, in a reasonable sense. We can stop talking about it.  The Turing Test has passed in another sense: it's over.  You can get computers to pass the Turing Test.  Big deal.

Wednesday, 4 June 2014

Yes For Independence, Because Eton

Last night I attended a debate about Scottish Independence at the University of St Andrews.  It was very good and well tempered on both sides.  

One of the arguments from the Yes side was that the UK Cabinet is dominated by Etonians. Obviously this is a major problem with Westminster politics, since one only needs to look at the education of the last few prime ministers.   I've examined the evidence and present it to you now, and you can see the point is completely definitive.  

  • Eton.  There you are.  David Cameron went to Eton. You see!  The point is proved. 

You might want to look away now, because is there any necessity to look back further?   You want to?  Ok.

  • Fettes College.  Well that is an Edinburgh school, but it has been called "Eton of the North". So maybe we can count it.  So yep, let's count it as showing the dominance of Eton, or at least posh public schools. 
Really, look away.  Stop reading.  The point is made!  
  • Rutlish Grammar School.  Perhaps not a surprise that John Major didn't go to a major public school.  Or even a public school.  But again, he's the Conservative exception to the rule.
  • Kesteven and Grantham Girls' School.  Maggie obviously couldn't go to Eton, as a woman. As the Conservative hate figure of recent times, she must have gone to a posh public school. Though oddly the school's history seems to suggest it was a grammar school.
Really, there is no need to carry on reading.  A couple of Tory Prime Ministers didn't go to Eton, and one of them was a woman so she couldn't.  
  • Eton.  There you are!  The point is confirmed.  Of course it's a bit embarrassing that this was a Scottish Prime Minister, Alec Douglas-Home, but perhaps we could pass a law banning First Ministers from having attended Eton.  
From now on backwards it's really easy to see the point, because it goes Eton (Macmillan), Eton (Eden), Harrow (Churchill), and Haileybury (Attlee).   More public schoolboys back to the Second World War.  So there you have it, from 1940 to 1964 and from 2010 to 2014, the UK has been governed by English public school pupils. 

Pay no attention to the period between 1964 and 2010, because it's irrelevant.  The fact that not a single Prime Minister of the United Kingdom in 46 years attended an English public school is totally irrelevant.

So if you don't like Eton, vote Yes for Independence. 

Author's note.  Like my previous piece, for avoidance of doubt I will state that this piece is not serious.  The point is to show that dominance of UK politics by Etonians, or even public school pupils, is not true. 

For avoidance of any more doubt, I don't think the educational makeup of the current cabinet is reason to vote against them.  The reason to vote against them is because you don't like their policies. I think that is a very good reason to vote against them: whether or not that is a good reason to vote for independence is another question. 

Tuesday, 20 May 2014

I Shall Vote Yes

I shall vote Yes because after three hundred years in the United Kingdom, Scotland is one of the richest countries in the world 
I shall vote Yes because I like the Queen
I shall vote Yes because I want to carry on using the British Pound 
I shall vote Yes because I don't want border posts between Scotland and England

I shall vote Yes because I want to keep watching the BBC
I shall vote Yes because I want to carry on charging non-Scottish British students for an education   
I shall vote Yes because I want Scotland to carry on spending 50% more on scientific research than it pays for 
I shall vote Yes because I hate Londoners greedily taking resources and not sharing them with Scotland
because I want Scotland to greedily take resources and not share them with Londoners  
I shall vote Yes because in 2010 I hated being ruled by a Westminster government that got as little as 35.6% of the vote in Scotland
I shall vote Yes because in 2010 I loved being ruled by a Holyrood government that got as much as 32.9% of the vote in Scotland   
I shall vote Yes because I want to be just like that country which is currently successful and not at all like that other country which I wanted to be like a few years ago  
I shall vote Yes because my definition of a bully is a unionist politician who doesn't do what I tell them to do 
I shall vote Yes because I want the Scottish Olympic team to have the same kind of success that the Scottish football team has 
I shall vote Yes because the word Yes is positive and the word No is negative    
I shall vote Yes because deep in my heart I believe that everything I like about the country I live in is Scottish and everything I don't like is British 

by Ian Gent
20 May 2014

Author's notes: 

This is based on "I shall vote Labour", a 1966 poem by Christopher Logue, though more directly on "I Shall Vote No" by A.R. Frith.

I am reliably assured that some people got through Frith's piece without realising it was not serious, so I will state a disclaimer.  My current expectation is actually that I will vote No.

Thursday, 19 December 2013

Ho, Ho, Who?

No, Virginia, I'm Not Santa
Interior, Parents' bedroom, Christmas Eve. 

Dad: Do you think you got away with it?

Mum: Yep.  She went straight to sleep.  Though when she was putting out the mince pie and the sherry, we got uncomfortably close to the bone... 

Interior, Child's bedroom, Christmas Eve, Fifteen Minutes earlier.

Virginia: Mum, it's him, isn't it?  I know Santa is Dr Who. 

M: Yes Virginia, of course Santa is Dr Who.

V: I worked it out!  It has to be.  How else could one person deliver presents to all the children in the world in one night?

M: They couldn't.

V: It's the TARDIS! He can go all over the world, street by street, and get into the TARDIS and go back in time a few minutes to go to the next street.  That way he doesn't have to travel at faster than light speeds or anything, he just goes back in time.  It's brilliant!

M: Well dear, don't spoil it for everyone.  They like to believe in magic.

V: Oh of course not, you should hear some of the silly things people at school believe!  But it's the TARDIS!   Obviously, how could Santa get all those presents for everyone into one sleigh? Ridiculous!  But the TARDIS, he just keeps filling it up and up and it never fills up, and off he goes with everyone's present.

M: What about a flat without a chimney like ours?  

V: Oh Mum, have you even watched Dr Who?  It's the sonic screwdriver!  That can open any lock in the world.  He just waves it and the door and walks in. And I know something else, too.

M: What's that, Virginia?

V: At school somebody told me Santa wasn't real because the Santa I met at the store looked different to the one at the town's Christmas lights switch-on.  But I know why... 

M: Ah, yes, umm ... 

V: He regenerated in between!  Obviously!  Of course he didn't look the same.  He'd probably been through ten regenerations and thousands of trips back in time going to different towns talking to children.

M: Time for bed dear. You wouldn't want to be awake when the Doct.. I mean when Santa comes. 

V: Night mum, I love you.  Don't worry, I won't tell anyone what I've figured out. 

Mum switches light out.  Switch back to scene in Parent's bedroom.  Wrapping last few presents.

Dad: She's a smart cookie. Just one little thing she didn't figure out.  Nobody ever talked about Santa and his police box.  Santa's TARDIS is still working and can look like a sleigh. 

M: His entire lifetime and all his regenerations as a Time Lord spent delivering presents on Earth.

D: And just for Christmas this year.  More than a thousand years to deliver to every house that celebrates Christmas.  At one minute a house.  And working 16 hours a day, no days off.  

M: Next year they sacrifice some other poor Time Lord to do it all again.  Why are they so cruel?

M&D: Here's to you, Time Lord.  We're thinking of you. 

Exterior, urban street decorated for Christmas.  TARDIS sound effect as sleigh materialises. Santa climbs out of sleigh and opens door with sonic screwdriver and walks in.  There is no spring in his step.

Monday, 28 October 2013

Gent's First Law of Putting Things Back

This is my small contribution to life.

When you can't find something, like say a pair of nail scissors or the widget that opens that thingy, you might look in ten different places before you find it.

If you are really organised, you put it away again after you've used it.

Where do you put it?

Not where you found it.

Not where you think the proper place is.

Put it back in the first place you looked for it.

Because that's where your mind tells you it will be, so the next time you lose it, that's where you'll look first.

That's Gent's First Law of Putting Things Back.

It's named immodestly because I'd like to be remembered for something.  And nobody's ever told me they heard this anywhere else.

Sunday, 27 October 2013

How The Grunch Stole C.S.

It's obvious what Birmingham University thinks about people like me.  It thinks this:
You're A Mean One, Mr Grunch
People like me don't do Computer Science.
That's the grunch.

It's also a bit of a surprise to me, because I've got some evidence I'm quite good at computer science. You know, what with me being a professor of Computer Science at a great university.

You know what a grunch is? Geekfeminism says a grunch is "the sensation a woman has when she ... is then reminded in some way that she is a woman first and a geek (or colleague, or writer, etc) a distant second."

I'm going to tell you some stuff I wanted never to tell you. Then I'm going to tell you why.

I didn't start puberty until I was 15.  

At some point around age 15 our English class had a debate on lowering the age of consent, so that people under 16 could have sex legally.  I felt I had a contribution to make, so I made it. It was pretty obvious the class thought it hilarious that somebody whose voice hadn't broken was talking about sex.

My voice broke sometime between about 15 to 17.  Being in sixth form (umm, grades 11-12 in US) with a breaking voice is not a barrel of laughs.  Though I don't remember my classmates being unduly unkind to me (not kind, you understand, but not cruel either.)

I didn't start shaving until I was about 17, after a year or more of ridiculous fuzzy face because I was too shy, embarrassed, scared, whatever, to find out how to shave.  

Some point around this time, I was the victim of a very mild sexual assault. I was riding home and some older boy somehow cornered me on my bike and touched my penis (umm, unless he made me touch his, I don't remember). 

At no time was I comfortable around girls, though this didn't matter very much because I never met any to speak of.   

How close did I get to "my first time" before I left Cambridge with a Maths degree at age 22? Well, let's see. If first base is mouth-to-mouth kissing, I can think of one time I nearly got to first base. That's it.

I literally thought I would never have sex, never get married. I learnt to live with that knowledge.

My entire romantic experience until I left university?  A very good friend of mine fell head over heels in love with me. He was a really nice guy but I was completely uninterested in men. Which meant it was a horrible experience for me, probably the most miserable time of my life. And it was worse for him, with his feelings unrequited. He tried to kill himself, which at least was some light relief for me.  Yes, I thought it was funny that I'd made somebody try to kill himself. Not who I wish I was, but that's where I was.  This whole episode made me borderline homophobic for some years, at least in my head - though I hope not in action.  (Now? I have wept tears of joy at same-sex couples I don't know getting married: I want others to have the joy I have in my heterosexual marriage.)

That's part of my life journey as a completely sexually inadequate teenager and young adult.

Why did I share this?

Because of this video from the University of Birmingham, aimed at recruiting students into Computer Science. Enjoy.

"Everyone remembers their first time, what was yours?"

The tag line for this video is "Everyone remembers their first time, what was yours?".  The opening title of the video is "My first time...".  Then we get people talking to camera saying things like "I was 16 or 17", "I was 15", "A very nervous 18", "I was primary school age, probably 9 or 10", "It was difficult at first but I soon got into it", "It was a lot harder than I first anticipated", "Well obviously I enjoyed it enough that I kept on doing it."  At the one minute mark some of the speakers giggle and they start talking about the first time they used computers or programmed them.

Oh, I get it, it's funny because they weren't talking about losing their virginity, they were talking about computers. Well, I'm not stupid, I knew all along they weren't talking about sex: it's a recruitment video.  But I'm meant to laugh along with them (remember the giggles?) that the apparent talk about first sex was just a joke.

I don't care if it's funny or not.

This is what I care about. Birmingam University thinks this:
People like me don't do Computer Science.
I don't think they consciously think that, but the following is what they must think (I'm using an amorphous "they" to mean the people responsible for making and releasing this video.)
  • We think equating first use of computers with first sex is funny
  • Everybody in computing is like us
  • Therefore  everybody thinks equating first use of computers with first sex is funny
Hey Birmingham, you know what? Not all of us are people like you. Not all of us had a good experience of first sex at an age before we are choosing a University.  Some people are like me: teenage sexuality was an embarrassment except when it was a cause of pain and misery. 

What if somebody is asexual, like maybe 1% of the population? What if somebody's first time was a criminal offence, maybe loving and consensual sex between a 16 year old and a 15 year old who are still together, and if anybody ever finds out the older partner faces being labelled a sex offender?  Oh, and you know that hilarious bit about being 9 or 10, to make us think it obviously is not about sex: today this card appeared on  postsecret.com : "I was nine. It was rape. This will no longer haunt me." 

I was nine. It was rape. This will no longer haunt me. 

Birmingham University risks making people like the above think this:
People like me don't do Computer Science.
That's the grunch.

Let me tell you, the grunch hurts. Just going over the video again to extract the quotes hurt me. You can of course rationalise the hurt by thinking it shouldn't have hurt me, I'm oversensitive, I've got no sense of humour, whatever you want. Fine. But it still hurt me. But that doesn't matter. Something very very big does matter.
People like me don't do Computer Science.
Maybe there's somebody like me age 17 going to see that video and get the grunch. Get the hit in solar plexus that says Computer Science is not for people like them. If they don't do Computer Science at Birmingham, that's not my loss. But if they don't do Computer Science at all, that's a loss for me. There's somebody been marginalised out of the discipline I love, which is horrible, and somebody who might have been brilliant (or at least like me good enough to be a Professor) doesn't make the contributions to C.S. that they could have made.

You think I'm overstating how much the grunch matters? Nope.

You want the scientific evidence? It's there in this Scientific American blog post.

You want a deeply personal story? It's there in this amazing post from Tim Chevalier.

I probably am overstating how much this video matters. It's a recruitment video from one Computer Science department.

This video may not matter, but the grunch really really really matters.

The grunch is stealing C.S.  It's stealing C.S. right now.

Please stop causing people the grunch.  Stop thinking that something that you find funny is - just because of that - not going to put somebody off C.S.

Over the last few weeks I've been doing a few small things to try to support women and other disadvantaged groups in tech. My wife has very kindly said that I'm being brave. Nonsense. This is the brave post.  Because it's the first one I'm scared to write. A lot of stuff above I have never told anybody, and I thought I would never tell anybody. Also, a lot of my friends disagree with me that this video should never have been used for recruitment (some think it is brilliant.)  So I know for a fact that my opinion is unpopular.

But my unpopular opinion remains that this video is a shame and a disservice to C.S. And a small example of the How The Grunch Stole C.S.

If this is the last thing you see, hit the "read more" button for credits, a postscript, and the most important links above in a useful bullet list.

Thursday, 24 October 2013

In which I am profoundly moved...

Two days ago I wrote about gender equality in constraint programming.

I am not joking when I say that I can't remember for some years being as excited about a work email as when I got the email from the CP 2014 organisers that they would institute an anti-harassment policy and look into childcare possibilities.  The very next day after my post.  I want to thank the CP conference organisers for their incredibly rapid and positive response.

Similarly I've been profoundly moved by the supportive comments made by others that I emailed. Not just supportive but coming up with other ideas of how we can improve, and reporting cases in the past where disadvantaged members of the community have been helped (which have not been publicised and for which I won't go into detail.)

As well as happy in itself, this makes me thrilled to part of such a welcoming community. I also hope we can be more open about some of these things, both so that people see how welcoming constraints is, and also so that people see that help might be available for them, who might otherwise not have thought to ask.

In terms of the harassment policy itself (of course still to be written), let me say right away that I absolutely agree that all forms of harassment are unacceptable, including based on gender, sexual orientation, race, disability, age, amongst others.  These should absolutely be included in the policy.

On facebook, a couple of people I respect criticised me for going public with my post instead of having this conversation in private.   (Obviously I won't be naming them here since they approached me in private, inasmuch as facebook is private.)  Certainly I would understand if the people I emailed felt hurt, though none of them have expressed this to me.

But here's the thing.  This is too important to do in private. I posted this in public because I wanted women to know that people in other communities would no longer stand by. Hence this twitter exchange:

But here's another thing.  I actually am approaching other people in private in other communities. I did feel however that I have to stand up in the community I'm in and say: yes, I am a part of a community which needs to be more publicly welcoming to everyone.   Part of my thrill over the last day has been in finding out just how welcoming constraints is, so there is less that needs to be changed and more that needs to be shouted about.

Obviously if anybody things anything I've said is wrong or unfair, or shares private information inappropriately, I do hope you will criticise me, either in public or private, and kudos to my friends for criticising me when that is what they felt.  

But finally, let me say one thing I completely agree with my friends about. It would be quite wrong if I got the credit for CP adopting policies.  At most I'd deserve credit for suggesting it. But anyone can suggest anything. Absolutely the relevant CP organisers deserve credit for making a move which will help women and other disadvantaged groups in our community.