Paging Dr. Watson: The Impact of Technology on Clinical Judgment

Last month, a provocatively-titled article in The Atlantic declared, “The Robot Will See You Now“. Starting with the story of IBM’s Watson going to medical school to learn to make diagnoses and treatment recommendations, it painted a future picture of medicine based on remote monitoring, clinical decision support, and above all, the cold, rigorous, seemingly-infallible power of Big Data.

“According to a growing number of observers, the next big thing to hit medical care will be new ways of accumulating, processing, and applying data.” Think: every primary care physician equipped with a Watson app to help them diagnose. Wearable sensors that track your blood pressure, heart readings, and numerous other indicators before you get to the doctor’s office. And as one venture capitalist predicts, computers and robots someday replacing 4 out of 5 physicians in the U.S.

Around the same time, I discovered a TED talk recorded in 2011 from Dr. Abraham Verghese, best-selling author and proponent of humanistic medicine, advocating for a very different kind of innovation:

“I’d like to introduce you to the most important innovation, I think, in medicine to come in the next 10 years,” Dr. Verghese declares, “and that is the power of the human hand.” Dr. Verghese goes on to describe the ritual of the physical exam–an age-old ritual of discovery and catharsis that has given way to something far less personal. He warns of a future “where the patient in the bed [becomes] an icon for the real patient who’s in the computer.”

“We seem to have forgotten — as though, with the explosion of knowledge, the whole human genome mapped out at our feet, we are lulled into inattention, forgetting that the ritual is cathartic to the physician, necessary for the patient — forgetting that the ritual has meaning and a singular message to convey to the patient.” -Dr. Abraham Verghese

So who’s right? Dr. Verghese’s message is simultaneously powerful and touching. But if we ascribe to his view, are we clinging to a romanticized notion of medicine at the cost of a less effective health care system? Are the two futures–one of “personalized” medicine and the other of medicine that is fundamentally personal–mutually incompatible? Or is there some middle ground we can aim for?

Déjà vu? The Unfulfilled Promise of EMRs

Turns out that somewhere 10,000 feet below the discussions of industry proponents and enamored health care executives, physicians have been debating a parallel of this question for quite some time. Not in response to the next robot-doctor, but to the replacement of good ol’ pen-and-paper notes with electronic medical records (EMRs).

According to a 2012 CDC report, the number of office-based physicians using EMRs has jumped from 18% in 2001 to 72% in 2012. EMRs were supposed to be the pill that saved us from an ailing, inefficient paper system. By digitizing all of our medical records, we could improve inter-provider communication, reduce errors, and finally solve the intractable problem of illegible physician handwriting. Indeed, much of the rapid adoption of EMRs in recent years has been driven by the Health Information Technology for Economic and Clinical Health (“HITECH”, har har) act of 2009, which provided incentives for physicians and hospitals to adopt EMRs. $27 billion of taxpayer money in incentives.

Unfortunately, a recent report by the RAND Corporation and subsequent reporting by the New York Times have highlighted some failures to achieve these promises. More and more people have begun seeing the whole thing as a scandal marred by industry influence, government complicity, and lack of evidence. Read a sample in this scathing account.

More directly relevant to our current discussion is the impact of EMRs on the end users (physicians) and the ultimate “beneficiaries” (patients). Driven like sharks to the frenzy of government money, EMR vendors rushed to spread their respective products, often without consulting the end user. As a result, physicians around the country began complaining about the burdens and limitations of EMRs, as Dr. Bob Watcher illustrates in a well-written post about the impact of EMRs on cognitive thinking:

“When I was on clinical service in July and read the notes written by our interns and residents, I often had no idea whether the patient was getting better or worse, whether our plan was or was not working, whether we need to rethink our whole approach or stay the course. In other words, I couldn’t figure out what was going on with the patient.” -Dr. Bob Wachter

The rigid, drop-down-menu style of EMRs may be impairing the ability of physicians to synthesize patient information. And that’s a scary thought.

Don’t Blame the Technology–Blame How We Use It

Interestingly, buried down below in the comments (credit goes to Dr. Leslie Kernisan for picking up on this) is a gem of a response from one Dr. Lawrence Weed–the same man who Dr. Wachter lauds in the opening of his post. “Our difficulty with Dr. Wachter’s analysis,” writes Dr. Weed, “is that he assumes the primary vehicle for clinical synthesis to be physician judgment. In reality, synthesis should begin before the exercise of judgment.”

In other words, why should old-fashioned physician thinking be the only way of synthesizing clinical information into a diagnosis? And why can’t technology like Watson be used to synthesize disordered clinical data, medical literature, insurance claims, and other data sources into meaningful clinical information that the physician can then use to make more informed clinical decisions?

Dr. Weed goes on to highlight an additional shortcoming of the current discourse around whether EMRs are good or bad: “Effective synthesis is tool-driven and process-driven.” (Emphasis mine.) And our debate over whether EMRs–or Watson–are “good” or “bad” for medicine ultimately addresses only half of the picture. Can we develop guidelines to ensure that the physician uses technology to inform rather than dictate the diagnosis? Can we establish processes that require the physician to continually elicit patient input into the treatment process?

The answers to those questions might be the key to determining the ultimate impact of technology on clinical reasoning and the wider field of medicine.

How Will It Help the Patient?

An extremely timely article published a few weeks ago in JAMA found 190 cases of missed primary care diagnoses in one year across two large health systems. Almost 80% of the errors were traced to “breakdowns in the physician-patient encounter”, with 56% of errors related to the medical history and 47% related to the physical examination. In an accompanying commentary, Dr. Newman-Toker estimates the error rate at about 0.1% of all primary care visits, which when extrapolated across the U.S. means at least 500,000 missed diagnostic opportunities each year.

(Update: This just in: diagnostic errors are the leading cause of successful malpractice claims, with 40% of diagnosis-related claims resulting in death.)

Just last week, a new survey by The Economist Intelligence Unit rank-ordered the activities in which “the need for retaining a role for human imagination or intuition [is] most critical.” The number one result is telling:

Source: Economist Intelligence Unit, 2013, "Smart Systems, Smarter Doctors: Humans and machines in healthcare".

Source: Economist Intelligence Unit, 2013, “Smart Systems, Smarter Doctors: Humans and machines in healthcare”.

These data point to an increasing urgency for solutions to improve the accuracy of our diagnostic capabilities , while simultaneously highlighting the tensions of emerging technologies such as Watson and their potential impact on clinical judgment. There is no simple answer to whether new technology will be good or bad for medicine. Rather, as our experience with EMRs shows, unlocking the promised benefits of technology will depend on our ability to foresee and mitigate a number of pitfalls:

  • Will the end-user (physicians, nurses, and other care practitioners) be consulted in every step of the design process, so that the technology can be smoothly integrated into workflows instead of creating onerous burdens?
  • Will the technology be designed to actually save money rather than driving ordering of unnecessary tests and services, as this recent Senators’ report accuses EMRs of doing?
  • Will medical education continue to stress the importance of clinical judgment, even with the proliferation of new tools at our disposal? Or will physician and care practitioners become increasingly dependent on the judgment of Dr. Watson?

Given the limitations of relying solely on old-fashioned clinical judgment, we must not be afraid to embrace the promise of technology such as EMRs and Watson. But we should be vigilant against unintended consequences these technologies may have on cognitive processes and patient care–this means rigorous evaluations of the impact of new technology, something that has been conspicuously missing for EMRs. And as we continue to debate the benefits of new technology on the physician, we should all keep in mind some words of wisdom from former CMS Administrator Don Berwick: “How will it help the patient?


Software Is Eating Health Care Too

Yesterday, at our annual firm-wide meeting, keynote speaker and board member Sanju K. Bansal shared some thoughts on the nature of innovation. Highlighting the role of technology, he cited a 2011 WSJ essay by Marc Andreessen titled, “Why Software Is Eating The World.


Relevant on so many levels. Source:

In it, Andreessen describes the “dramatic and broad technological and economic shift in which software companies are poised to take over large swathes of the economy”. “Health care and education, in my view,” writes Andreessen, “are next up for fundamental software-based transformation.” (Which, incidentally, are the two industries that my current employer focuses on.)

Ignoring Andreessen’s apparent miss with his market-is-undervaluing-Internet-companies theory (Two days ago, Facebook’s shares were below 50% of its IPO price again), I found the most prescient part of his argument (at least for health care) embodied in the following paragraph:

“Today’s leading real-world retailer, Wal-Mart, uses software to power its logistics and distribution capabilities, which it has used to crush its competition. Likewise for FedEx, which is best thought of as a software network that happens to have trucks, planes and distribution hubs attached. And the success or failure of airlines today and in the future hinges on their ability to price tickets and optimize routes and yields correctly—with software.”

The next wave of technological disruption in health care isn’t necessarily electronic medical records (EMRs), even if you could get doctors to agree to use them (which we don’t seem to be doing a good job of doing). It’s data analytics. What are the drivers of unnecessary variation (and thus, cost)? Where are the bottlenecks that are slowing ED throughput? Which procedures produce the best outcomes for each particular patient, not in the generalizable randomized controlled trial sense, but for the specific patients in your specific locale entering your specific institution?

I’ll provide a more concrete example using an area I’m currently focusing on: device implants. This subset of supplies are also often called “physician preference items”—because other than the peculiar preferences of each particular surgeon, there doesn’t appear to be much else evidence in support of using one brand over another. In fact, the decisions are more often governed by which sales representative the surgeon met during residency training, or who their golfing buddies are. Given the power of physicians to take their business elsewhere if pressured to change brands, hospitals face immense challenges trying to get physicians to standardize to a less costly brand or more clinically effective model.

That second goal is particularly challenging, given that we currently don’t really know which brands work better than others. The traditional academic literature has failed us here. Perhaps more accurately, the problem is that we’ve been asking the wrong question. The question isn’t which brands work better than others universally. It’s which brands work better for which types of patients under what circumstances.

The Patient Centered Outcomes Research Institute (PCORI), established by the ACA, takes some important steps toward promoting so-called “comparative effectiveness research” that recognizes the messiness of real life medicine. However, for the hospital executive, what will be more useful will be real-time data on which brands are providing the best outcomes for his/her hospital’s patients. And that’s where the data analytics comes in.

Physicians (and surgeons especially) may be at times stubborn, but I believe they are first and foremost committed to their patients’ outcomes, and they are strong believers in data. If you can set up a well-running Materials Management Information System (MMIS), link it to your EMR system, and tack on the spending data so you can evaluate cost-effectiveness, you’ll have a powerful tool to win over your surgeons while identifying the best care practices for your patients.

Most hospitals are far from the kind of systems interoperability described here—in fact, it’s been described as the “Holy Grail” of physician preference items. But it’s coming—in the end, software will eat health care too.

Faulty Defibrillators and Broken Hip Implants: A Terrible Case of Déjà vu

My blogging has been a little on the lighter side of late, since I’ve recently been focused on learning about some unfamiliar but fascinating topics on the business of health care through my new job at the Advisory Board. But I came across a highly relevant news article today, and decided to take the opportunity to share about what I think is a dangerous and often-overlooked problem in our health care system.

The Wall Street Journal today reported on device manufacturer St. Jude Medical Inc., which recalled its Riata defibrillator in 2010 after reports that the defibrillator cables were breaking through their insulation from the inside out. However, documents collected by the FDA reveal that they had known about the problem as early as 2005, and even received reports from physicians between 2006-2009 of the problems. Later in the day, my work-related literature search led me to a 2008 New York Times article reporting on a two other devices that were recalled due to defects: Zimmer Holdings’ Durom cup, a socket used in hip replacements, and Sulzer Orthopedics’ hip implant. The stories outlined in the two articles sound strikingly similar: a few doctors noticed problems early, sounded the alarm, were notified by the company that they were isolated incidents or due to poor surgical technique…and no one could provide systematic evidence suggesting otherwise.

They could have, if the U.S. had a device registry. A device registry is essentially a surveillance system that tracks every device model implanted into every patient, along with the name of operating surgeon and the technique used. The aggregate data can provide early alerts on particular devices that are exhibiting abnormal rates of problems. National registries have been set up in other countries, including Australia, Sweden, Norway, and Britain, where the risk of revision surgeries (repeat surgeries due to device failures or improper technique) is less than half as high as the risk in the U.S. Despite scattered efforts in the U.S. to set up device registries by systems such as Kaiser Permanente and the Hospital for Special Surgery in New York, efforts to set up a national registry have failed for a myriad of reasons such as low physician participation, lack of legal governance, and potentially insidious relationships between orthopedists and device manufacturers.

There is an additional benefit to having a device registry beyond having an early warning alert system to defective devices: having such a large data set may help us take much-needed steps toward evaluating the clinical effectiveness of devices. As Ventola reviews in this 2008 article, we currently face huge challenges in trying to understand and compare just how effective different medical devices are. Although the FDA has rigorous testing requirements for drugs (that’s why it takes an average of 12 years for an experimental drug to go from lab to market), its process for medical devices is a bit more…lax. Basically new devices fall into one of three categories:

  1. Low risk products (e.g. bandages, splints, drapes): No premarket review requirements at all;
  2. Moderate risk products (an incremental change to an existing product): Requires a premarket notification process where “you provide notice to the FDA that your product is substantially equivalent to a product they’ve already seen.” (Yeah, think about that for a second. This is the route that 95-98% of devices take.)
  3. High risk products: Requires clinical trials.

Source: Ventola, C.L. 2008. Challenges in evaluating and standardizing medical devices in health care facilities. P&T 33(6):348-359.

It’s no wonder that very few clinical trials exist on the effectiveness of devices, and the majority of those that do are shoddy-quality studies conducted by the device manufacturers themselves. When you consider that next to the findings of a landmark 2008 CBO study that roughly half of the growth in health care spending over the past several decades can be attributed to new technology, you start to wonder why we don’t more rigorously evaluate whether new, expensive devices actually give us more bang for our buck, as the UK does through their ironically named “NICE” (National Institute for Health and Clinical Excellence), which evaluates the cost-effectiveness of all new technologies.

(Incidentally, it appears that even with this guidance, physicians will still be physicians and refuse to listen to NICE’s recommendations.)

However, I’m of the optimistic opinion that physicians WILL listen to evidence that their particular device of choice is harming their patients. And so a device registry, whether national or private, seems like a much-needed solution to a currently overlooked problem.

“He has been at it for three decades,” the 2008 Times article writes of veteran orthopedic surgeon Dr. Dorr, “long enough to say that history is repeating itself because this country does not gather evidence of how patients fare.”

Tragically, four years later, it appears that history is still repeating itself.