Archive for February, 2011

On the Contrary… my dear Watson! Or is it?

February 18, 2011 Leave a comment

Much of the country tuned in to Jeopardy this week and were surprised to find a twist in the game. Two of the game’s biggest winner’s, each earning six figure’s in cash prizes throughout their appearances on the show, were back in the spotlight competing head-to-head. To challenge the top two contestants in Jeopardy history, a third competitor, Watson, was brought to the stage to upend them. The unexpected twist – Watson is an IBM supercomputer with the ability to search through massive amounts of data at unprecedented speeds. In a test of human vs. artificial intelligence, IBM had sent a robot to do the dirty work that countless other human competitors failed to do so many times previously.

In a two day event designed to showcase the advances in IBM technology, IBM hoped to display just how smart Watson was by attempting to defeat the “smartest of the smartest” while the whole world watched it all go down. True to their word, IBM delivered an incredibly smart computer that dominated the game in nearly every aspect. Or did it?

Watson managed to chime in on almost every clue, delivering near perfect success. It certainly demonstrated just how much data Watson had access to, how complex the algorithms must have been to sort through it all, and how fast it was able to process it all by returning correct answers at the bat of an eye. However, there were a variety of questions that Watson remained silent on and a few answers that he gave that were completely illogical (i.e. Toronto, for an answer – a final jeopardy answer no less – as a response to a clue that specifically asked for a U.S. city).

It seemed as though Watson was more adept at returning correct responses for clues that had specific dates, times, names, and/or places within the answer than he was at delivering responses for the questions that IBM had led us all to believe he was capable of – the clues that were complex and required analytical thinking. Furthermore, Watson would reveal a variety of options that he thought could be the correct response and show the percentage of how confident he was in that answer. In some cases, he would reveal the correct response associated with the least amount of confidence in its truth. Regardless of answering he demonstrated that sometimes his inability to come up with the correct response was not due to the lack of information available to him, but that he was unable to determine which information was the most relevant and which was not. In other words, the answer was out there for him to retrieve it, he was just unable to fully comprehend the question and decipher the information it came across to associate and return the correct response.

One of the most significant aspects of the game that stood out to anyone “hoping” for a human to beat Watson, was that Ken tried buzzing in on almost every answer that Watson did. Unfortunately Watson almost always beat him to it. In terms of fairness, I think the final scores were not reflective of how much “smarter” Watson was than the two human competitors. Without reaction time factored in, I believe the margin between the final scores would’ve been a lot less. For Ken to determine the right answer and react fast enough to push the button before Watson would mean his reaction time would have to be faster than the speed at which Watson can send an electronic signal which triggers the buzzer. It would be interesting to know what that time lapse was and if the game had gotten to the point where the two human competitors had to buzz in immediately on every clue as soon as they were allowed (regardless of whether or not they already knew the correct response) just to stand a chance at buzzing in before Watson

As a last concern, the arbitrary “wagering” that Watson made during daily double attempts and in final jeopardy rounds was off-putting. If IBM can design a supercomputer that is capable of sorting through million’s of pages of content in a split second and determining an answer based upon a complex set of algorithms wouldn’t you think that they could develop an algorithm that didn’t just wager “arbitrary” amounts of money somewhere in the range of zero dollars and whatever total Watson had at the moment? Perhaps a simple addition to the algorithm which allowed him to compare the current cash totals of his opponents and wager an amount to keep him within a couple thousand dollars in the event that he got the answer wrong? Instead, with a lead of several thousand dollars over his opponents and no possibility of losing the lead going into final jeopardy on day 1, we got a wager of a couple hundred dollars. The selling point of the robot was his ability to analyze information as it would be disseminated to a human yet IBM didn’t allow Watson to be proactive in his wagering strategy, basing his decision on his answer accuracy or the scores of his opponents.

Watson the supercomputer? On the contrary my fellow readers! Watson was merely fast at sorting through data and buzzing in. Although, all skepticism aside, I am impressed with his ability to do just that and can see a very practical use for him in the real world with the example that IBM gave us… perhaps in the medical industry looking up any and all symptoms associated with a patient and sorting through previous cases documented online and giving suggestions for treatment. There is just no doubt in my mind that I want a human doctor making the final decision.