I was excited for this man vs. machine Jeopardy battle and then I almost completely missed it. If it hadn’t been for a headline I saw on the internet machine, I wouldn’t have been tuning in on Tuesday night for round two of a 3-day battle. The first of two games was stretched over two days so that IBM could get their share of air time. The show did feel like a bit like a long IBM commercial, but it’s Jeopardy, not some movie you paid 12 bucks to see, so it’s not the end of the world. Listening to the IBM people was pretty interesting as well, because I can’t fathom the technology and ingenuity it would take to make this computer. I still don’t really understand how a telephone works. I don’t want to try to figure out how computers do anything, let alone dominate trivia shows.
The game itself probably didn’t live up to expectations. The problem is, the computer (Watson) is just too good with buzzing in for an answer. The only thing that slows its ringing in is if it has some doubt about the answer it will give. This gives the humans (Ken Jennings and Brad Rutter) time to slip in an occasional answer. The show today, I’d say Watson rang in first at least 90% of the time. And, it was almost always correct. I’d say the most interesting thing about Watson is that you can see its top-3 answers for the questions with weighted confidences. The first choice is usually correct and has a confidence in the 95% range. The other choices are often odd answers with very low confidence. This is because the hardest thing for Watson to do, is decipher the question. Those tricky Jeopardy punsters.
This isn’t really a fair fight, and I felt a little badly for Jennings and Rutter. Although, not really for Ken Jennings. I think he was a little bit of a Jeopardy creation. Obviously, the guy is very good, but his streak of wins was uncommonly lucky. I don’t want to say there was foul play, but Jeopardy could certainly manage the outcome a bit once the streak was building and they started getting incredible amounts of attention. If you watch Jeopardy, you know that not all contestants are made equal, and I wonder if Ken got more than his share of B-teamers during his run? Just a little conspiracy theory. When Jennings is up against other dominant players, like Rutter, he is at-best on their level, and probably in the case of Rutter, just slightly below. I guess it’s like Joe-D’s streak, just because he hit in 56 straight, doesn’t mean he’s the best hitter who ever lived.
Getting back to it not being a fair fight, high level Jeopardy is all about ringing in, and the human participants don’t have much of a chance, so they sit there and look like idiots while Watson streaks to a huge lead. For the purposes of the game, they’ve really just proven that the computer has faster reflexes. Which is still incredibly impressive. I can’t overstate how cool the computer is. I’m someone who gets excited when a calculator spits out a correct answer. This is unbelievable. My main complaint with Watson is that if I had to get an answer right, one answer, to a question that wasn’t necessarily impossible, I’d still take Jennings or Rutter over the machine.
When Watson misses, it misses big. Like in Final Jeopardy, a lay-up of a question about U.S. Cities produced the answer, “Toronto” from Watson. Luckily, the computer only risked about 1,000 dollars and maintained its huge lead, but on the one of the easiest questions all night Watson wasn’t even in the ballpark. So, the practical application of this machine may still be far off. Also, why can’t we make any advances in computerized speech? This thing sounds like the computer in War Games. If I was at IBM, I’d be a little embarrassed.
The match-up ends tomorrow night with the 2nd and final game. With the cumulative scoring, and Watson’s complete lack of greed and off the charts buzzer skills, I don’t see how it can be beaten. If you didn’t catch it in action though, you should definitely give it a peek. At the very least Jennings may squirt a few tears, he was looking pretty whiny during Double Jeopardy.