Watson a game-changer for science

Here’s the clue: It is quick on the buzzer and stuffed with the equivalent of one million books, and it can beat you at Jeopardy!

The answer: What is Watson?

Watson is the IBM supercomputer that became a whiz at Jeopardy!, the long-running television quiz show. In a February 2011 showdown, the brainy machine beat out the two best-ever human Jeopardy! champs.

Sure, Watson can sift through trillions of pages of text per second, but did scientists spend all that time building a machine just to win bragging rights for a game show?

Not really.

Watson’s triumph marked a major technical achievement for scientists. That’s because in order to win, the computer had to “understand” what it was reading.

Computers have been doing smart things for years. But until now, no machine has been able to do what humans do everyday — make sense of the words people use when they talk to each other

Getting machines to comprehend language has long been a goal of scientists working in a field of research called artificial intelligence, or AI. But human language is full of variety and vagueness — something computers don’t deal with well. One phrase may have several different meanings. Or, in the case of Jeopardy!, a simple clue shown on a screen and read by host Alex Trebek can be phrased in many different ways. In order to understand the meaning of the words, a computer needs to learn context, slang and all the other nuances people use in everyday conversation, or natural language.

That’s been the challenge for both computers and their designers, says IBM scientist David Ferrucci. He knows because he spent four years directing a team of scientists to prepare Watson for its match.

Round one

Watson is not the first high-profile game-winning computer. In 1997, IBM developed a chess-playing supercomputer called Deep Blue. In a match with Garry Kasparov, the reigning world chess champion at the time, Deep Blue won.

But it’s much easier to teach a computer to play chess than it is to teach a computer to play Jeopardy! Chess has strict rules. The game’s rules and strategies can be coded into computer language that tells the machine exactly what to do and when. At each turn in the chess game, Deep Blue’s computer brain could scan through millions of possible moves to pick the best one.

Getting a computer to understand what a human is asking is much harder to do. The wording is tricky, and the clues on Jeopardy! are often loaded with jokes or puns. And some words have several meanings. A simple phrase, such as “the bat hit the window” can easily trip up a computer, Ferrucci says. “You have to know whether the clue is referring to the flying creature or to a baseball bat.”

Humans are naturals at putting words into context. They can do so by hearing or seeing how the word is used in a sentence. Computers can’t do this.

To solve this problem, Ferrucci created sets of instructions, also known as algorithms. Each works like a flowchart, taking Watson step-by-step through the process of analyzing a sentence.

The process is similar to what students do in grammar, Ferrucci says. “Watson learned how to take a sentence and break it down into its parts — subject, verb and so on,” he says. Because sentence structures vary, the researchers wrote hundreds of algorithms to cover all the bases.

Figuring out the various parts of the sentence told Watson how the words were related. The computer then needed a way to sort out the words’ meaning.

Building confidence

The scientists crammed Watson with information. Pages from encyclopedias, dictionaries, history books, biographies and textbooks were fed into the system. Details on movies, music, popular books and current events were added to the database, too.

The team also wrote new algorithms that showed Watson how to look for connections among the data. Unlike a search engine that will turn up anything that contains a keyword — like “bat” — Watson could understand whether it was looking for information on flying mammals or wooden clubs.

Once Watson could grasp the meaning of the words in a clue, it needed a system to form a response. It also needed a way to gauge the accuracy of its answer.

“If you buzz in and get the answer wrong, you’ll lose the dollar value for that question,” Ferrucci says. So Watson had to evaluate when the chance of being wrong was too high to risk buzzing in an answer.

The scientists wrote more algorithms. For each clue, Watson carried out hundreds, even thousands, of searches to consider the many possible answers. When it came up with a potential answer, Watson combed its massive database again, looking for evidence to support each answer. During this evaluation process, Watson analyzed hundreds of pages of text to check the dates, times, people and places the question called for. If the answer “made sense,” Watson would buzz in in less than three seconds.

A team of two dozen scientists worked for four years to develop the instructions needed to show the machine how to analyze sentences and generate and evaluate answers.

For each of those steps, the algorithms evolved over time. As the scientists added new algorithms and rewrote old ones to help the machine better analyze what it was reading, the computer got faster and smarter.

Meanwhile, Watson’s system got smarter about using the algorithms. The scientists used a method called statistical machine learning to help Watson learn from its own training. This allowed Watson to use statistics — probabilities that are based on data — to balance and weigh its own algorithms based on which ones contribute to the right answer.

“Over time, the machine learns on its own and is getting the right answer more often than not,” Ferrucci says.

Watson received 10,000 old Jeopardy! questions to use for practice. When the computer spit out an answer, the researchers told it whether the answer was right or wrong. Through this rigorous training, the machine learned when a particular algorithm was useful.

As Watson got faster, its system got larger. Watson’s brainpower runs on 90 powerful servers, or master computers, each with a 32-core processor. This setup allows the machine to run many programs at the same time. And with 15 trillion bytes of memory to carry out the computations, Watson can perform trillions of operations per second. That’s as powerful as 6,000 to 10,000 desktop computers all working together, Ferrucci says.

Final Jeopardy!

So, with all this computer power, does Watson understand natural language as well as humans do? Not even close, Ferrucci says. “Humans do all this with a brain that fits in a shoebox and is powered by a tuna-fish sandwich and a glass of water.”

Still, Watson does well enough. It beat Ken Jennings, who won a record 74 games in a row on Jeopardy!, and Brad Rutter, another former Jeopardy! champion. Researchers are now finding ways to use the technology behind Watson to tackle problems outside of Jeopardy!

Herbert Chase, who teaches medicine at Columbia University in New York City, has a team of medical students training Watson to answer doctors’ questions about patient care. The students feed Watson information on a patient’s symptoms, lab tests and family history. Watson takes these “clues” and looks for all the possible diseases, illnesses or treatments that apply. The students then grade Watson’s performance.

“Doctors have literally dozens of questions by the end of a typical day,” Chase says. “For example, I might ask, ‘Which diabetes drug is best for high school students?’”

With its massive computing power, Watson can dig through information in medical textbooks, reference materials — even blogs that have been uploaded to its memory — to help doctors make connections they might have otherwise missed.

Watson’s ability to sort through information and find connections might also be useful in fields such as law and finance, Ferrucci says. “There’s a lot of valuable knowledge in all those textbooks, reference books, blogs and news. Watson can dig deep into all this material to find the evidence needed to make decisions.”

Scientists still have work to do to make computers that can think and understand language as well as humans. But some people, including Jeopardy! host Alex Trebek, are already speculating on the abilities of the next generation of Watson.

“If this was just the tip of the iceberg,” Trebek says, “then our viewers have much to look forward to…Watson, the new ‘Jeopardy!’ host!”
 

Watch IBM supercomputer Watson and former Jeopardy! champs Ken Jennings and Brad Rutter prepare to match wits. Like humans, Watson can understand and analyze natural language. Courtesy Jeopardy Productions, Inc.

Watch Watson answer Daily Double and Final Jeopardy! questions. Courtesy Jeopardy Productions, Inc.

Power Words:

Algorithm: A set of instructions written for the computer in coded language that tells the computer what to do.

Artificial intelligence: A branch of science that studies how to make machines think like humans. Also known as AI.

Bytes: A basic unit of stored information capable of designating a single character, such as a letter or number.

Database: An organized collection of information stored in a computer.

Memory: In computing, the information stored in electronic form, or the total information-holding capacity of an electronic device or computer.

Operation: In computing, an elementary task, such as adding two numbers.

Program: A series of instructions for a computer.

Server: A master computer that manages all of the computers, terminals, data and programs that may be shared by the devices that are connected to it.

Pun: A funny play on words in which one word or phrase takes on more than one meaning. For example: I was going to look for my missing watch, but I never could find the time.

Statistics: A branch of mathematics that uses collected data to express the likelihood of something. For example: 5 out of 10 children brush their teeth.