Chatting with a Chatbot

The widely reported claim that the “Eugene Goostman” chatbot passed the Turing Test has been debunked.

I am not a computer scientist or even a techie (and I certainly have no expertise in Artificial Intelligence); but I am a lawyer and journalist and so I am used to asking questions.  Not just difficult or loaded questions, but all sorts of questions, depending on what purpose I have in asking them.

So I thought I would chat with this chatbot this lunchtime.

I started by asking the most basic question “what is your favourite colour”.

It gave a cute answer about asking instead about its favourite size and shape. So I took the word “shape” and asked what it meant.

It did not know.

I then asked about Turing, and got a glib answer about him being a “pioneer”. So I asked what “pioneer” meant.

It did not know.

And so I carried on – not trying to ambush the bot with unusual words or hard questions, but simply taking words it chose to use and asking it to explain them.

And not once could it explain any word that itself had chosen to use.

A Ukrainian 13 year old speaking English as a second language would be able to explain (in some fashion) the words he or she was choosing to use.

From the hype I was expecting something impressive, even if not of the level to pass the Turing Test; but, on any view, this was just embarrassingly bad.  Not only did it have no “idea” what it was being asked, it did not even have any “idea” what it was saying itself.

Leave a Comment