Will IBM’s Watson Kill Your Career?

IBM’s Watson made major headlines last year when it trounced its human rivals on Jeopardy. But Watson isn’t just sitting around spinning trivia questions to stump the champs: IBM is working hard on taking it into a series of vertical markets such as healthcare, contact management and financial services to see if the system can be used for diagnosing diseases and catching market trends. Does this spell the end of your BI career? Not really, but it does raise some interesting thoughts and issues.

I had the chance to see Watson up close and personal at the St. Louis Gateway to Innovation conference in April. There, IBM had a kiosk modeled after the Jeopardy show. You picked information categories and received answers, for which you needed to type the matching questions.

It was interesting, because you got to see how Watson “thinks” and formulates its replies. I barely started typing before it finished figuring out its (correct) question. The kiosk didn’t contain the actual Watson computer: it was just a remote interface back to IBM Research, where the system really resides. You can see a picture of part of the kiosk here, and how it judges among three different possible responses, along with assigning confidence values to each:

For a better idea of how exactly Watson reasons and learns from its data, you can watch this YouTube recording of Dr. David Ferrucci, IBM Fellow and Principal Investigator for Watson, who spoke at the Future of Health Technology Summit last month. Watson’s programming relies on three different kinds of reasoning: temporal, statistical, and geospatial. 

Now, I consider myself a fairly decent Jeopardy player: when I watch the show on TV, I can usually blurt out the right question more than half the time. But that’s sitting in the comfort of my own living room, and not at a keyboard. When it came to standing in front of the Watson kiosk, I choked. Big time. I couldn’t even start typing before it had figured things out. It was embarrassing. Now I know how those Jeopardy champs felt.

Watson is far from perfect: On the televised Jeopardy show last year, Watson flamed out on the Final round, when it confused the location of the Chicago airports and gave its answer “What is Toronto.” Even I got that one right.

At the kiosk, I began to see how Watson works: it examines a lot of different possibilities, and then starts adjusting its confidence intervals bit by bit. It looks for patterns, for relevance, for data relationships; not unlike what your typical BI analyst does when presented with a mountain of data.

Retraining

The rub is that Watson is only as good as its human trainers are—and there’s a considerable amount of training involved. There are 220 people involved in the Watson project, a large staff devoted to thinking up ways to stump the machine. They needed more than a week to load in a new database for each Jeopardy software build.

For the medical apps currently under development, it still takes the team several days to retrain Watson. Part of the problem: understanding how to retire old data, or assign lesser confidence intervals as you learn new things about the subjects under consideration. Knowledge is not a static situation, and we tend to take for granted how we learn things. But it also represents a hard problem for the Watson team.

At the IBM Impact conference in Vegas in May, I sat in on a session where Bob Madey, one of Watson’s developers, discussed the team’s next steps. He mentioned healthcare specifically and showed how a typical patient diagnosis would work with the system. IBM is collaborating with Wellpoint, a major healthcare provider, to improve the process for evaluating a potential patient who either calls or walks into a facility.

Dr. Josko Silobrcic, a medical scientist at IBM Research, told me: “New information is added to Watson in several different ways. On the patient side, the healthcare provider adds their electronic records to the system. On the evidence side, it happens by accessing the latest medical sources such as journal articles or Web-based data and clinical trials.”

He added: “Another way is through training and re-training of Watson. Because Watson leverages machine learning, it needs all three of these methods to evaluate and weigh the new information.”

Part of this training involves the use of known sources, whose answers become a benchmark for ensuring that Watson indeed understands things correctly.

Wellness

Ashok Chennuru, the Director of Technology for Wellpoint, predicted that a team of 45 people could eventually work on Watson. People involved in the project, he said, possess “combination of skills of IT like QA engineers, content experts in clinical and pharmaceutical areas, and also statisticians. We want to add outcome-based researchers and comparative effectiveness research to the team eventually.” IBM expects availability for the first apps probably by early 2013.

Someone in the audience at Impact asked how long it would take for IBM to develop the Watson equivalent of a WebMD.com (or the holographic doctor on Star Trek: Voyager), with anyone able to query its medical data. Madey was cagey about answering, but such a thing isn’t entirely outside the realm of possibility. I started thinking of the HAL9000 computer that powers the spaceship in 2001: “Dave, that’s a really good idea.”

IBM is also working with Wellpoint and Memorial Sloan Kettering to use Watson in cancer care, providing detailed diagnostic and treatment options based on the latest research.

“Watson will not have any patient ID data; we would have problems with compliance with various regulations,” said Chennuru. “But family history is included, and our job is to parse and structure a lot of the transcription notes and other information. For example, a patient may say they are a non-smoker but there are other indications about previous smoking history or other conditions. The more information you have about the patient, the better off you are with your treatment options.”

IBM is also interested in healthcare projects beyond Watson. For example, in Haiti the company is collaborating with Colleagues in Care. IBM SmartCloud for Social Business virtually connects 200 doctors, nurses, business professionals and other medical workers and volunteers from around the globe, all of them feeding treatment options, clinical pathways, and Haiti-specific best practices into an online medical knowledge system.

For the healthcare and other verticals, IBM envisions some kind of B.I.-as-a-service offering in the not-too-distant future. Madey mentioned one possibility for how IBM would charge for such a service: IBM will take 40 percent of the calculated savings from its use. So if Watson is helping to diagnose diseases, the hospital and IBM will come to some agreement as to the cost savings from this activity. The client only pays if Watson delivers.

“It took us a while to work out with IBM the payment arrangements,” said Chennuru. “We had our internal actuaries to do the modeling and justify the project to take into a lot of factors into consideration.” (IBM will probably rely on other pricing models, as well.)

This differs from usual computer services types of deals where you pay by the CPU minute or the amount of storage or other resources consumed. That wouldn’t do for Watson, because of the massive resources involved in processing its natural language queries and storing all the data it needs to figure out the context of the query and possible answers to questions. And one other thing: IBM will own all the data that is fed into Watson. You just get the results, and that’s it. That’s not going to be an attractive arrangement for everyone.

What It Means for You

So before you start thinking about rebooting your B.I. career for a post-Watson era, consider that in the short term your knowledge (particularly about your business practices and how your company’s products and services relate to one another) is probably going to become a lot more valuable than not: all that information you’ve amassed over the years will need to be translated, encoded, schematized, and plotted out before it can be used by a system like Watson. IBM (and others) will need you as subject-matter experts to chart out the landscape ahead.

It was fun to try out Watson, even though I did so poorly. Now I know how those fellow carbon-based life forms felt on Jeopardy last year: it was humbling just to try my hand at a couple of questions. But it’s also exciting to see where IBM is taking the project, and I suspect we’ll hear more about this curious machine sometime soon. In the meantime, you don’t have to worry about Watson replacing you—at least, not yet.

 

Images: IBM (top), David Strom

Post a Comment

Your email address will not be published.