Saturday, February 12, 2011

Jeopardy is Elementary for IBM's Watson

Mark this coming Monday on your calendar: history will be made.  On Valentine’s Day, as you are sitting down to dinner with the love of your life, make sure that your PVR is set to record Jeopardy!.  From the 14th to the 16th of February, Jeopardy! will feature its first non-human contestant.  IBM’s most ambitious creation, an intelligent computer, will compete on TV’s longest running game show against its two top champions: Ken Jennings, who won on seventy-four consecutive outings and Brad Rutter, the show’s top money earner.

IBM’s super computer was affectionately named Watson, after the company’s founder (not Sherlock’s sidekick).  Watson is just a jumble of silicon, wires, and coding, yet it represents a major step towards artificial intelligence.  By today’s standards, it is not a significant accomplishment to build a computer that stores massive amounts of information, and accesses it on a whim.  What sets Watson apart from other super computers is its ability to understand a question and generate an appropriate answer.  Of course, to succeed in Jeopardy!, it must comprehend a question and generate the correct answer, but it amounts to the same thing.

The English language has so many subtleties that go unnoticed by people that use it.  It is extremely difficult to teach a computer such nuances.  A person picks up language with practice and intuition over the course of many years.  People are blessed with common sense.  Do you know how many logical statements must be fed to a computer for it to have as much common sense as an adult?  Leading computer scientists and artificial intelligence engineers believe the number is around two hundred million.  A logical statement such as, “An organism that dies can never return to life,” is just one of 200,000,000 that need be entered into a computer in order for it to converse with the same common sense that humans take for granted in one another.

The Watson project was a four-year-long labour of love, but it did not start out so promisingly.  In the early going, Watson was fed a steady diet of encyclopaedias, newspapers, science journals, and other content that could prove useful when competing on the challenging game-show.  With basic algorithms inserted to comprehend a clue and yield an answer, it would do so, and also supply the confidence it had in the answer, say 75% for fairly confident.  Early on in the project, its success rate for answering Jeopardy! calibre questions was only 10%.  Although he and I could have a competitive match with this kind of proficiency, he would be demolished by a typical Jeopardy contestant.  A contestant like Ken Jennings can answer above 85% of questions correctly.

The reason for Watson’s poor performance early on was not due to insufficient information.  It was simply a matter of not understanding the question.  Jeopardy questions have puns and expressions that people grasp instantly, but sail right through the mainframe of a computer (in one port and out the other).  The way IBM solved this problem is intriguing.  They implemented a procedure known as machine learning.

Machine learning sounds threatening.  It is reminiscent of Terminator and Matrix movies in which machines take over.  Basically, a machine can teach itself things by means of pattern recognition.  You can feed a machine a bunch of data, and it will be limited to these specific bits of information if it does not connect the dots between them.  If, on the other hand, you ask the computer to make generalizations based on several data points or examples, its learning becomes exponential.  IBM fed Watson episode upon episode of previous Jeopardy answers with their corresponding correct questions, and asked it to use these data points when answering future questions.  By means of interpolation (connecting the dots of these data points), Watson’s success rate jumped from 10% to 60% almost instantaneously.

After a long and very interesting tuning process, Watson’s success rate jumped above 80%, and Jeopardy executives deemed him ready to compete on the air against its two top champions.  However, the decision to put Watson on primetime was not arrived at lightly.  Watson had to audition like any other contestant.  Watson has no advantages over his human competitors.  There will be no special treatment.  It is not wired to the internet.  Because it is deaf, it receives questions at the same moment as the humans, but via text messages.

This major event may seem like a publicity stunt for both IBM and Jeopardy!, and of course, it is.  But it is also much more than that.  It echoes the famous 1997 chess match between Kasparov and IBM’s Deep Blue.  Deep Blue defeated the chess grandmaster, and the event was regarded as a significant step towards A.I. 

This new computer challenge represents a much larger step of the sort.  While chess requires much skill for a human, it is a simple game for a computer to play, as there is one goal (checkmate the king before you are checkmated) for the computer to focus on, and some basic rules that must be followed.  Jeopardy! questions require a certain interaction on the part of Watson, and the topic of conversation is basically infinite.

Still, I do not believe that Watson is intelligent.  Don’t get me wrong – Jeopardy! is probably the most challenging trivia show in existence.  Someone such as myself, with limited trivia skill, could walk away from Who Wants to be a Millionaire? with thousands of dollars, but would get laughed off of Jeopardy!.  Yet, the blue-screened clues always point to one answer.  Watson may prove superior to the brightest humans when it comes to solving well-defined problems.  I do not think this is the true mark of intelligence.

Watson cannot dabble in philosophy.  He can only seek solutions to problems to which there is one answer.  This is a marvellous skill, though not as impressive as what humans do when they discuss the unknown.  Questions with infinite possible solutions require an additional ingredient, known as creativity.  Watson cannot paint an original painting, or compose an original music composition.

I believe that computers will achieve human-level intelligence at some point in the 21st century.  Watson will have been one major step towards that singularity: that crucial point of no return.

No comments: