Way back in 1997, IBM’s Deep Blue computer beat chess grandmaster Garry Kasparov. Now, IBM’s Watson computer has creamed the legendary Ken Jennings at Jeopardy.
What does this prove? It depends on whom you ask.
Mastermix notes that, “players are surprised at how big a role button management plays in winning or losing a round. In the few minutes of the Watson game that I watched, it was pretty clear that Watson was excellent at pressing the button at exactly the right moment if it knew the answer, which is more a measure of electromechanical reflex than human-like intelligence.”
Jennings himself thinks it says that human emotions are a liability in competition.
Watson has lots in common with a top-ranked human Jeopardy! player: It’s very smart, very fast, speaks in an uneven monotone, and has never known the touch of a woman. But unlike us, Watson cannot be intimidated. It never gets cocky or discouraged. It plays its game coldly, implacably, always offering a perfectly timed buzz when it’s confident about an answer. Jeopardy! devotees know that buzzer skill is crucial—games between humans are more often won by the fastest thumb than the fastest brain. This advantage is only magnified when one of the “thumbs” is an electromagnetic solenoid trigged by a microsecond-precise jolt of current. I knew it would take some lucky breaks to keep up with the computer, since it couldn’t be beaten on speed.
During my 2004 Jeopardy! streak, I was accustomed to mowing down players already demoralized at having to play a long-standing winner like me. But against Watson I felt like the underdog, and as a result I started out too aggressively, blowing high-dollar-value questions on the decade in which the first crossword puzzle appeared (the 1910s) and the handicap of Olympic gymnast George Eyser (he was missing his left leg). At the end of the first game, Watson had what seemed like an insurmountable lead of more than $30,000. I tried to keep my chin up, but in the back of mind, I was already thinking about a possible consolation prize: a second-place finish ahead of the show’s other human contestant and my quiz-show archrival, undefeated Jeopardy! phenom Brad Rutter.
One bit of consolation for the puny human:
“Watching you on Jeopardy! is what inspired the whole project,” one IBM engineer told me, consolingly. “And we looked at your games over and over, your style of play. There’s a lot of you in Watson.”
Glenn Reynolds thinks the Singularity is nigh. Stephen Gordon, too:
A simple definition of Singularity is “that point at which greater-than-human intelligence arises.”
[…]
The human champion describes Watson as a Terminator. It never gets tired, bored, has stage fright, gets cocky, or intimidated. It just keeps coming… answering general knowledge questions better than the best Jeopardy players on the planet. Scary right? Then, the reporters go Utopian and suggest that Watson and its decendents could be used to diagnose illnesses. It could be Dr. House without the attitude. Exciting, right?
Donald Sensing isn’t impressed:
Strip out Watson’s blinding speed, and it is no smarter than human beings at all. Watson, for all its engineering impressiveness, simply did only what computers have always done: collate at blinding speed (and compute mathematical probabilities to choose an answer). It does not matter that Watson was not connected to the Internet since its mass-memory unit holds 16 Terabytes of data, processed by a 2,880 processor core. As my own computer professor said (many years ago!), “A computer is just a high-speed moron.” There is nothing about Watson that I have read so far that obviates that observation.
[…]
So what does Watson really prove? From a technical, engineering and programming perspective, it’s an amazing achievement with enormous potential for a wide range of applications ranging across broad multi-disciplinary subjects and problems. As for the Jeopardy game, there’s less than meets the eye. Count it as a proof-of-concept exercise. What it did not do was reasons abstractly. It just collated amazing amounts of information very rapidly. But we already knew that computers are faster than we are for specified tasks. That’s why we build them to begin with.
Peter David agrees:
Predictably, the computer is currently smashing the humans. Except Watson is hardly the HAL 9000. It’s more like the Google 9000. It’s a pocket calculator with voice recognition. Feed it enough proper nouns and it can call up information fed into it by countless programmers (which means Jennings and Rutter aren’t really facing a single opponent; they’re fighting hundreds of human minds). But give it anything vaguely abstract and it falls apart.
The Final Jeopardy category was U.S. Cities. The answer was that this city had two airports, the largest of which was named after a World War II hero, and the second largest after a famous World War II battle. It took me three seconds to figure it; Jennings and Rutter probably less than that. Obviously it’s Chicago (O’Hare and Midway).
Watson’s answer? Toronto, an answer so wrong on every level that it didn’t even fit the category. If a human had been given Toronto as one option of a multiple choice, he would have rejected it based on the knowledge that Toronto isn’t a US city.
If the “Jeopardy!” writers were so inclined, they could produce entire boards of the type of questions that we saw in “Final Jeopardy” and Watson would just sit there flummoxed while the humans’ superior reasoning power would run roughshod over it.
Computers can react faster than humans, but they can’t think faster than humans. They can’t think at all, because there’s more to thinking than just regurgitating facts.
Steve Hamm falls somewhere in between. Turning to the infamous airport flub:
David Ferrucci, the manager of the Watson project at IBM Research, explained during a viewing of the show on Monday morning that several things probably confused Watson. First, the category names on Jeopardy! are tricky. The answers often do not exactly fit the category. Watson, in his training phase, learned that categories only weakly suggest the kind of answer that is expected, and, therefore, the machine downgrades their significance. The way the language was parsed provided an advantage for the humans and a disadvantage for Watson, as well. “What US city” wasn’t in the question. If it had been, Watson would have given US cities much more weight as it searched for the answer. Adding to the confusion for Watson, there are cities named Toronto in the United States and the Toronto in Canada has an American League baseball team. It probably picked up those facts from the written material it has digested. Also, the machine didn’t find much evidence to connect either city’s airport to World War II. (Chicago was a very close second on Watson’s list of possible answers.) So this is just one of those situations that’s a snap for a reasonably knowledgeable human but a true brain teaser for the machine.
The mistake actually encouraged Ferrucci. “It’s goodness,” he said. Watson knew it did not know that right answer with any confidence. Its confidence level was about 30%. So it was right about that. Moreover, Watson has learned how the categories work in Jeopardy! It understands some of the subtleties of the game, and it doesn’t make simplistic assumptions. Think about how Watson could be used in medicine, as a diagnostic aid. A patient may describe to a doctor a certain symptom or a high level of pain, which, on the surface, may seem to be an important clue to the cause of the ailment. But Watson may know from looking at a lot of data that that symptom or pain isn’t the key piece of evidence, and could alert the doctor to be aware of other factors.
Watson’s achievements in sorting through the questions and processing information against the best of the best human competitors is simply remarkable. And, presumably, computers will get much better at this sort of thing in relatively short order. (Although, man, I’d have sworn that the Deep Blue-Kasparov match was much more recent than 1997!)
Is it anything approaching the complex reasoning skill that even an average human mind can bring to the table? Not hardly. But Watson is a better raw data storage and retrieval unit than any human is likely to ever be; and, again, computers are only going to get more powerful.
Update (Alex Knapp): My own take on the Jeopardy match is here, and this is my takeaway:
Truth be told, this isn’t a contest between “man and machine” — it’s a contest between two Jeopardy contestants and a team of IBM programmers. To me, the programming and algorithm discovery needed to create Watson are what’s impressive. And that’s all a product of the human mind — not the computer.
When a computer is capable of teaching itself how to play Jeopardy, is limited to buzzing at human speeds, and then beats two champions. I’ll be impressed by the computer.
For now, I’m impressed by the all too human programmers.
Furthermore, it’s worth noting that while everyone LOVES to talk about Deep Blue beating Kasparov in 1997, you’ll note that far fewer people talk about Kasparov’s match against Deep Junior in 2003. It was a more complicated computer, but Kasparov fought it to a draw (1-1-4). Indeed, if you go to chess boards, there are lots of discussions about how to get around computer advantages and defeat computer AI at chess. Moreover, Kasparov himself has developed a variant of chess called Advanced Chess, which takes advantage of human strategic superiority and computer tactical superiority to produce absolutely beautiful games of chess.
Computers may be getting smarter, but humans are, too.
