Paging Dr. Watson: The Impact of Technology on Clinical Judgment

Last month, a provocatively-titled article in The Atlantic declared, “The Robot Will See You Now“. Starting with the story of IBM’s Watson going to medical school to learn to make diagnoses and treatment recommendations, it painted a future picture of medicine based on remote monitoring, clinical decision support, and above all, the cold, rigorous, seemingly-infallible power of Big Data.

“According to a growing number of observers, the next big thing to hit medical care will be new ways of accumulating, processing, and applying data.” Think: every primary care physician equipped with a Watson app to help them diagnose. Wearable sensors that track your blood pressure, heart readings, and numerous other indicators before you get to the doctor’s office. And as one venture capitalist predicts, computers and robots someday replacing 4 out of 5 physicians in the U.S.

Around the same time, I discovered a TED talk recorded in 2011 from Dr. Abraham Verghese, best-selling author and proponent of humanistic medicine, advocating for a very different kind of innovation:

“I’d like to introduce you to the most important innovation, I think, in medicine to come in the next 10 years,” Dr. Verghese declares, “and that is the power of the human hand.” Dr. Verghese goes on to describe the ritual of the physical exam–an age-old ritual of discovery and catharsis that has given way to something far less personal. He warns of a future “where the patient in the bed [becomes] an icon for the real patient who’s in the computer.”

“We seem to have forgotten — as though, with the explosion of knowledge, the whole human genome mapped out at our feet, we are lulled into inattention, forgetting that the ritual is cathartic to the physician, necessary for the patient — forgetting that the ritual has meaning and a singular message to convey to the patient.” -Dr. Abraham Verghese

So who’s right? Dr. Verghese’s message is simultaneously powerful and touching. But if we ascribe to his view, are we clinging to a romanticized notion of medicine at the cost of a less effective health care system? Are the two futures–one of “personalized” medicine and the other of medicine that is fundamentally personal–mutually incompatible? Or is there some middle ground we can aim for?

Déjà vu? The Unfulfilled Promise of EMRs

Turns out that somewhere 10,000 feet below the discussions of industry proponents and enamored health care executives, physicians have been debating a parallel of this question for quite some time. Not in response to the next robot-doctor, but to the replacement of good ol’ pen-and-paper notes with electronic medical records (EMRs).

According to a 2012 CDC report, the number of office-based physicians using EMRs has jumped from 18% in 2001 to 72% in 2012. EMRs were supposed to be the pill that saved us from an ailing, inefficient paper system. By digitizing all of our medical records, we could improve inter-provider communication, reduce errors, and finally solve the intractable problem of illegible physician handwriting. Indeed, much of the rapid adoption of EMRs in recent years has been driven by the Health Information Technology for Economic and Clinical Health (“HITECH”, har har) act of 2009, which provided incentives for physicians and hospitals to adopt EMRs. $27 billion of taxpayer money in incentives.

Unfortunately, a recent report by the RAND Corporation and subsequent reporting by the New York Times have highlighted some failures to achieve these promises. More and more people have begun seeing the whole thing as a scandal marred by industry influence, government complicity, and lack of evidence. Read a sample in this scathing account.

More directly relevant to our current discussion is the impact of EMRs on the end users (physicians) and the ultimate “beneficiaries” (patients). Driven like sharks to the frenzy of government money, EMR vendors rushed to spread their respective products, often without consulting the end user. As a result, physicians around the country began complaining about the burdens and limitations of EMRs, as Dr. Bob Watcher illustrates in a well-written post about the impact of EMRs on cognitive thinking:

“When I was on clinical service in July and read the notes written by our interns and residents, I often had no idea whether the patient was getting better or worse, whether our plan was or was not working, whether we need to rethink our whole approach or stay the course. In other words, I couldn’t figure out what was going on with the patient.” -Dr. Bob Wachter

The rigid, drop-down-menu style of EMRs may be impairing the ability of physicians to synthesize patient information. And that’s a scary thought.

Don’t Blame the Technology–Blame How We Use It

Interestingly, buried down below in the comments (credit goes to Dr. Leslie Kernisan for picking up on this) is a gem of a response from one Dr. Lawrence Weed–the same man who Dr. Wachter lauds in the opening of his post. “Our difficulty with Dr. Wachter’s analysis,” writes Dr. Weed, “is that he assumes the primary vehicle for clinical synthesis to be physician judgment. In reality, synthesis should begin before the exercise of judgment.”

In other words, why should old-fashioned physician thinking be the only way of synthesizing clinical information into a diagnosis? And why can’t technology like Watson be used to synthesize disordered clinical data, medical literature, insurance claims, and other data sources into meaningful clinical information that the physician can then use to make more informed clinical decisions?

Dr. Weed goes on to highlight an additional shortcoming of the current discourse around whether EMRs are good or bad: “Effective synthesis is tool-driven and process-driven.” (Emphasis mine.) And our debate over whether EMRs–or Watson–are “good” or “bad” for medicine ultimately addresses only half of the picture. Can we develop guidelines to ensure that the physician uses technology to inform rather than dictate the diagnosis? Can we establish processes that require the physician to continually elicit patient input into the treatment process?

The answers to those questions might be the key to determining the ultimate impact of technology on clinical reasoning and the wider field of medicine.

How Will It Help the Patient?

An extremely timely article published a few weeks ago in JAMA found 190 cases of missed primary care diagnoses in one year across two large health systems. Almost 80% of the errors were traced to “breakdowns in the physician-patient encounter”, with 56% of errors related to the medical history and 47% related to the physical examination. In an accompanying commentary, Dr. Newman-Toker estimates the error rate at about 0.1% of all primary care visits, which when extrapolated across the U.S. means at least 500,000 missed diagnostic opportunities each year.

(Update: This just in: diagnostic errors are the leading cause of successful malpractice claims, with 40% of diagnosis-related claims resulting in death.)

Just last week, a new survey by The Economist Intelligence Unit rank-ordered the activities in which “the need for retaining a role for human imagination or intuition [is] most critical.” The number one result is telling:

Source: Economist Intelligence Unit, 2013, "Smart Systems, Smarter Doctors: Humans and machines in healthcare".

Source: Economist Intelligence Unit, 2013, “Smart Systems, Smarter Doctors: Humans and machines in healthcare”.

These data point to an increasing urgency for solutions to improve the accuracy of our diagnostic capabilities , while simultaneously highlighting the tensions of emerging technologies such as Watson and their potential impact on clinical judgment. There is no simple answer to whether new technology will be good or bad for medicine. Rather, as our experience with EMRs shows, unlocking the promised benefits of technology will depend on our ability to foresee and mitigate a number of pitfalls:

  • Will the end-user (physicians, nurses, and other care practitioners) be consulted in every step of the design process, so that the technology can be smoothly integrated into workflows instead of creating onerous burdens?
  • Will the technology be designed to actually save money rather than driving ordering of unnecessary tests and services, as this recent Senators’ report accuses EMRs of doing?
  • Will medical education continue to stress the importance of clinical judgment, even with the proliferation of new tools at our disposal? Or will physician and care practitioners become increasingly dependent on the judgment of Dr. Watson?

Given the limitations of relying solely on old-fashioned clinical judgment, we must not be afraid to embrace the promise of technology such as EMRs and Watson. But we should be vigilant against unintended consequences these technologies may have on cognitive processes and patient care–this means rigorous evaluations of the impact of new technology, something that has been conspicuously missing for EMRs. And as we continue to debate the benefits of new technology on the physician, we should all keep in mind some words of wisdom from former CMS Administrator Don Berwick: “How will it help the patient?


Software Is Eating Health Care Too

Yesterday, at our annual firm-wide meeting, keynote speaker and board member Sanju K. Bansal shared some thoughts on the nature of innovation. Highlighting the role of technology, he cited a 2011 WSJ essay by Marc Andreessen titled, “Why Software Is Eating The World.


Relevant on so many levels. Source:

In it, Andreessen describes the “dramatic and broad technological and economic shift in which software companies are poised to take over large swathes of the economy”. “Health care and education, in my view,” writes Andreessen, “are next up for fundamental software-based transformation.” (Which, incidentally, are the two industries that my current employer focuses on.)

Ignoring Andreessen’s apparent miss with his market-is-undervaluing-Internet-companies theory (Two days ago, Facebook’s shares were below 50% of its IPO price again), I found the most prescient part of his argument (at least for health care) embodied in the following paragraph:

“Today’s leading real-world retailer, Wal-Mart, uses software to power its logistics and distribution capabilities, which it has used to crush its competition. Likewise for FedEx, which is best thought of as a software network that happens to have trucks, planes and distribution hubs attached. And the success or failure of airlines today and in the future hinges on their ability to price tickets and optimize routes and yields correctly—with software.”

The next wave of technological disruption in health care isn’t necessarily electronic medical records (EMRs), even if you could get doctors to agree to use them (which we don’t seem to be doing a good job of doing). It’s data analytics. What are the drivers of unnecessary variation (and thus, cost)? Where are the bottlenecks that are slowing ED throughput? Which procedures produce the best outcomes for each particular patient, not in the generalizable randomized controlled trial sense, but for the specific patients in your specific locale entering your specific institution?

I’ll provide a more concrete example using an area I’m currently focusing on: device implants. This subset of supplies are also often called “physician preference items”—because other than the peculiar preferences of each particular surgeon, there doesn’t appear to be much else evidence in support of using one brand over another. In fact, the decisions are more often governed by which sales representative the surgeon met during residency training, or who their golfing buddies are. Given the power of physicians to take their business elsewhere if pressured to change brands, hospitals face immense challenges trying to get physicians to standardize to a less costly brand or more clinically effective model.

That second goal is particularly challenging, given that we currently don’t really know which brands work better than others. The traditional academic literature has failed us here. Perhaps more accurately, the problem is that we’ve been asking the wrong question. The question isn’t which brands work better than others universally. It’s which brands work better for which types of patients under what circumstances.

The Patient Centered Outcomes Research Institute (PCORI), established by the ACA, takes some important steps toward promoting so-called “comparative effectiveness research” that recognizes the messiness of real life medicine. However, for the hospital executive, what will be more useful will be real-time data on which brands are providing the best outcomes for his/her hospital’s patients. And that’s where the data analytics comes in.

Physicians (and surgeons especially) may be at times stubborn, but I believe they are first and foremost committed to their patients’ outcomes, and they are strong believers in data. If you can set up a well-running Materials Management Information System (MMIS), link it to your EMR system, and tack on the spending data so you can evaluate cost-effectiveness, you’ll have a powerful tool to win over your surgeons while identifying the best care practices for your patients.

Most hospitals are far from the kind of systems interoperability described here—in fact, it’s been described as the “Holy Grail” of physician preference items. But it’s coming—in the end, software will eat health care too.