Docs Vastly Outperform Computer Algorithms in Diagnostic Accuracy

Greg Slabodkin | Health Data Management | October 13, 2016

The maxim “doctors know best” is getting support from a new study led by Harvard Medical School researchers that shows physicians made a correct diagnosis more than twice as often as 23 commonly used symptom checkers—websites and apps that help patients with self-diagnosis. The study’s results, published October 10 in the journal JAMA Internal Medicine, are being touted as the first direct comparison between human-made and computer-based diagnoses.

Ateev MehrotraIn the study, doctors vastly outperformed symptom checkers, arriving at a correct diagnosis 72 percent of the time versus 34 percent of the time for the digital platforms. In addition, 84 percent of physicians listed the correct diagnosis in the top three possibilities, compared with 51 percent for the symptom checkers. “People are out there using them,” says Ateev Mehrotra, MD, associate professor in the Department of Health Care Policy at Harvard Medical School and senior investigator for the study, who estimates that “there are more than 100 million uses per year of these symptom checkers” among consumers.

“While in the future computers might outperform physicians, these computer algorithms that are being used on a regular basis clearly do not,” adds Mehrotra. Researchers evaluated the diagnostic accuracy of 23 symptom checkers using 45 clinical vignettes, which included the patient’s medical history but had no physical examination or test findings. The study also compared the diagnostic performance of physicians with symptom checkers for those same clinical vignettes using Human Dx, a web/app-based online platform on which doctors generate differential diagnoses for clinical vignettes...