Man Beats Machine in Correct Diagnosis

Health care systems are increasingly looking to integrate smart computers into their day-to-day operations. The idea is they can help physicians better and more quickly diagnose and treat whatever may be ailing a patient.

This week a new partnership was announced between the German-based Siemens Healthineers corporation and IBM Watson Health to integrate their Watson “supercomputer” into all hospitals and other health care settings as quickly as possible worldwide.

In a head-to-head comparison, human doctors got the diagnosis right 72 percent of the time, compared to 34 percent for the apps.

Watson reportedly can help health care professionals navigate the growing volume and diversity of health data, an aging global population, the increasing prevalence of chronic diseases, changes in healthcare payment models, and the digitization and consumerization of healthcare, company officials stated.

Yet, while having access to all that knowledge and research can be helpful, physicians on their own still do a far better job of diagnosing patients correctly, according to a new study.

In a head-to-head comparison, human doctors with access to the same information about medical history and symptoms as was put into a symptom checker got the diagnosis right 72 percent of the time, compared to 34 percent for the apps. The 23 online symptom checkers — some accessed via websites and others available as apps — included those offered by Web MD and the Mayo Clinic in the U.S. and the Isabel Symptom Checker in the U.K.

“The current symptom checkers, I was not surprised, do not outperform doctors,” senior author Dr. Ateev Mehrotra of Harvard Medical School in Boston said in a statement. But in reality, computers and human doctors may both be involved in a diagnosis, rather than pitted against each other, Mehrotra told Reuters Health.

Related: Health Apps: Get a Second Opinion

Using a web platform called Human Dx, 45 clinical vignettes were distributed to 234 physicians. Doctors could not do a physical examination on the hypothetical patient or run tests; they had only the information provided. Doctors submitted their answers as free text responses with potential diagnoses ranked in order of likelihood.

Compared to putting the same information into symptom checkers, physicians ranked the correct diagnosis first more often for every case. The results were published in a research letter in JAMA Internal Medicine.

[lz_ndn video=31500638]

“In medical school, we are taught to consider broad differential diagnoses that include rare conditions, and to consider life-threatening diagnoses,” Dr. Andrew M. Fine of Boston Children’s Hospital, who was not part of the new study, told Reuters. “National board exams also assess our abilities to recognize rare and ‘can’t miss’ diagnoses, so perhaps the clinicians have been conditioned to look for these diagnoses,” he said.

Physicians aren’t always right, though; the thought is that their outcomes will only improve with computers backing them up. What patients can additionally take away from the findings is that the symptom checkers they find online aren’t always right either, and shouldn’t be considered a substitute for an exam by a health care provider when there are concerns.