Mind Matters News and Analysis on Natural and Artificial Intelligence
Medical and healthcare,Male doctor or medical students or surgeon,anesthesiologists using digital tablet during the conference,Health Check with digital system support for patient,background banner
Medical and healthcare,Male doctor or medical students or surgeon,anesthesiologists using digitals tablet during the conference,Health Check with digital system support for patient,background banner
Featured image: ER needs info ASAP/havea niceday, Adobe Stock

Why Was IBM Watson a Flop in Medicine?

Robert J. Marks and Gary S. Smith discuss how the AI couldn’t identify which information in the tsunami of medical literature actually MATTERED

Economist Gary N. Smith and Bradley Center director Robert J. Marks continued their discussion of many things AI, including John Searle’s Chinese Room, but talk turned to why Watson couldn’t turn its famous 2011 Jeopardy win into usefulness in medicine:

Show Notes: The Holy Grail of Artificial Intelligence: Gary N. Smith wonders whether AI will ever achieve common sense

Excerpt:

Robert J. Marks: Do you know the current status of IBM Watson? I’ve heard some bad things.

Gary N. Smith: They’ve been trying to go into all sorts of things with mixed success and one of the most hopeful things was that they would be able to revolutionize medical care, health care. And it’s not worked because they could look up symptoms of various diseases and they could look up cures for various diseases and they could look up medical articles. But they don’t understand which are more meaningful than others, which medical articles are reasonable and which are bull and so a lot of doctors have become disillusioned with Watson. And a lot of hospitals have literally pulled the plug.

Just last year, the IBM Health Initiative laid off a bunch of people because of this disillusionment.

Robert J. Marks: Yeah, I was really excited about IBM Watson being able to help with cancer. I know that in my field and probably in your field also that there are hundreds of papers published daily in our discipline and there’s no way we can keep up with them.

Gary N. Smith: And there’s no way to tell which ones are real and which ones are just BS…

Robert J. Marks: I was really excited because you know, these physicians and they have this a cancer patient and they have a certain number of systems and you can go to this IBM Watson and they can read them the symptoms and try to let them understand—of course, Watson doesn’t “understand”—but try to tell Watson what these symptoms are. And Watson digs through all these papers that are published every day, which are thousands per month and tens of thousands over the last decade and actually grabs the latest results and brings it to the physicians and says, Try this!

And, like you say, even if it was able to do that, there is no way to test the viability and the validity of the results that Watson gives. I understand, as you do, that IBM Watson is kind of going “toes up” I understand that MD Anderson Cancer Center, the project fails and it is shut down, and it’s looking for other vendors now.

Is the failure due simply to IBM Watson’s inability to understand or is it due to business practices? I’ve heard both. I don’t know what the status of the failure is.

Gary N. Smith: I don’t know how much to weight them but I am pretty confident that a lot of it is due to the fact that computers don’t understand what sense to make of such things and how to sort through sense and nonsense.

Robert J. Marks: Now, I’ve talked to people who say, hey, it’s like googling. When you google something, you have to be pretty specific with the keywords that you enter. Maybe these doctors and physicians just don’t know the right keywords to enter. What would be your reaction to that?

Gary N. Smith: I don’t think it’s that. I think the problem goes much, much deeper in the sense that, like I’ve been saying, all that computers could come back with is output. They don’t know which output is relevant and which output is irrelevant.

Robert J. Marks: I’ve also heard that they mismanaged it. I know as an engineer, you get a good result and you think the world’s going to beat a path to your door and use your invention, and boy that doesn’t work! You need to interface and you need to get down and dirty with the people that are actually applying it.

You mentioned that one of your colleagues said that IBM Watson was a “fraud.” I’ve got a quote from a venture capitalist, Chamath Palihapitiya [pronunciation here]. He is the CEO of an investment firm called Social Capital and he pulled no punches. He said, “Watson is a joke, just to be honest.” And then he said, in very nice words, that IBM Watson appeals to stupid people. Here’s what he said: “I think that IBM is excellent at using their sales and marketing infrastructure to convince people who have asymmetrically less knowledge to pay for something.” I think somebody with “asymmetrically less knowledge” is somebody that’s stupid, right? He said it so nicely though…

Gary N. Smith: In 2017, AI was the Marketing Word of the Year and all these companies are saying, We do AI because it sounds so cool and sexy, and it’s often just a marketing gimmick.

[4:20– 9:10]

This image has an empty alt attribute; its file name is image.jpeg

Palihapitiya makes it sound rather as if Watson has been abandoned in the Unclaimed Baggage compound at the international airport in Toronto… 😉 If so, call that Jeopardy’s revenge, if you like. Or perhaps Toronto’s.


Further reading on IBM Watson:

Why did Watson think Toronto was in the USA? How that happened tells us a lot about what AI can and can’t do, to this day

Why an AI pioneer thinks Watson is a “fraud.” The famous Jeopardy contest in 2011 worked around the fact that Watson could not grasp the meaning of anything.

and

IBM Watson is not our new computer overlord. AI help, not hype: It won at Jeopardy (with specially chosen “softball” questions) but is not the hoped-for aid to cancer specialists. (Robert J. Marks)

Earlier discussions between Robert J. Marks and Gary Smith:

Why an AI pioneer thinks Watson is a “fraud.” The famous Jeopardy contest in 2011 worked around the fact that Watson could not grasp the meaning of anything.

Can AI combat misleading medical research? No, because AI doesn’t address the “Texas Sharpshooter Fallacies” that produce the bad data.

AI delusions: A statistics expert sets us straight. We learn why Watson’s programmers did not want certain Jeopardy questions asked.

and

The US 2016 election: Why Big Data failed. Economics professor Gary Smith sheds light on the surprise result.

Featured image: ER needs info ASAP/haveaniceday, Adobe Stock


Mind Matters News

Breaking and noteworthy news from the exciting world of natural and artificial intelligence at MindMatters.ai.

Why Was IBM Watson a Flop in Medicine?