Since the CheXNet paper came out in November 2017 I have been communicating with the author team. I'm finally ready to review the paper. Some of the things I found out surprised me.
I just wanted to do a quick follow up to my recent blog post, which discussed the performance metrics I think might be appropriate for use in medical AI studies. One thing I didn't cover was the reason we might want to use multiple metrics, or the philosophy behind choosing the ones I did. So today, … Continue reading The philosophical argument for using ROC curves
2017 was cool. Medical AI progressed apace, the AI community grew up some and got a bit creative, and I made some predictions that mostly held up to vague scrutiny.
A couple of weeks ago, I mentioned I had some concerns about the ChestXray14 dataset. I said I would come back when I had more info, and since then I have been digging into the data. I've talked with Dr Summers via email a few times as well. Unfortunately, this exploration has only increased my concerns about the dataset.
I have been doing a fair bit of thinking about the blog, and wanted to clue you all in on some changes. I had intended to finish my "The End of Human Doctors" series before moving on to anything else, and I even let some really cool topics pass me by to maintain continuity. It … Continue reading Blog plans for 2018
Deep learning research in medicine is a bit like the Wild West at the moment; sometimes you find gold, sometimes a giant steampunk spider-bot causes a ruckus. This has derailed my series on whether AI will be replacing doctors soon, as I have felt the need to focus a bit more on how to assess … Continue reading Do machines actually beat doctors? ROC curves and performance metrics
So, the big news in medical AI research is that the Stanford ML group under Andrew Ng has released a paper on chest x-ray interpretation that claims human performance at identifying pneumonia. First up, very cool! Second up, I have concerns. Vague and not so vague discomforts. So, after having a few days to wrap … Continue reading Quick thoughts on ChestXray14, performance claims, and clinical tasks.
Today I want to look at two papers which tell us something very useful about medical AI, particularly if we are trying to predict the future of medicine.
Today we continue looking at breakthrough medical deep learning research, and review a major paper from Stanford researchers that reports "dermatologist level classification of skin cancer", published in Janurary 2017. As a reminder, a major focus of this dive into the state of the art research will be barriers to medical AI, particularly technical barriers. This … Continue reading The End of Human Doctors – The Bleeding Edge of Medical AI Research (Part 2)
More than any other part of this blog series, what we talk about today will have the most impact on whether machines are going to replace doctors anytime soon. We are going to start exploring the cutting edge of research in medical automation. In the previous articles in this series, we simply assumed deep learning can automate medical tasks. … Continue reading The End of Human Doctors – The Bleeding Edge of Medical AI Research (Part 1)