Epistemic Arrogance. Case Records of the Massachusetts General Hospital. Diagnosis is Stochastic, NOT Deterministic

This post was spurred by my reading the July 24^th NEJM case records, “A Man with Cough, Dyspnea, and Hypoxemia.”  When I was still using twitter, I mused that the discussants have too high a rate of getting the right diagnosis. This implies that there is no stochastic or aleatory uncertainty in diagnosis; that if you have the requisite information and reasoning process, you can connect the dots, every single time, to the final diagnosis. In this view, there is no randomness to diagnosis, no inherent and irreducible probabilistic (or stochastic, or aleatory) uncertainty. Of course, this is pure codswallop: stochastic uncertainty is what makes diagnosis so hard.

You cannot say that the result in any case was not possible with the information presented, but with a population of cases you can make inferences about whether the percentage of correct discussant diagnoses is improbable. The fact that the discussant gets the right diagnosis almost 100% of the time (I have tabulated the % correct for over 10 years of these cases – it rounds to 100%) in the CRMGH (Case Records of the Massachusetts General Hospital) is like finding too little variability in a research study (or  studies; or Bernie Madoff’s annual returns) and concluding that the results (returns) were manipulated. We use our well-justified first principles assumption of stochasticity, find that there is not enough variability, and conclude that something is being manipulated to remove the aleatory uncertainty. (I use stochastic, aleatory, and probabilistic herein as interchangeable adjectives for uncertainty.) And so must it be in the CRMGH. How the aleatory uncertainty is being removed is anybody’s guess and any insider’s knowledge, I suppose. Possibilities include both implicit and explicit cues. An implicit cue may be the selection of, say, a specialist in heritable cardiomyopathies as the discussant in a case where the final diagnosis is a genetic cardiomyopathy. Explicit cues may be that somebody tells the discussant the final diagnosis at the outset or tells them to revise their discussion if they get it wrong in draft. Hospitals are not closed systems either. People hear if there was a case of Creutzfeld-Jacob diagnosed recently, and the director of the microbiology lab knows if a particular rare species was recently cultured. Tales of rare diagnoses spread, as they are the currency of clinical acumen. I can’t say that any of these mechanisms -- from subtle queueing to explicit cheating -- are occurring. I only know that the near 100% hit rate means that randomness has been removed from an inherently random process.

Numeracy in Medicine: Clinical Wagers to Estimate Probability in Diagnosis

As has become my custom, I offered a wager on rounds the other day. An nonagenarian had gastrointestinal hemorrhage. He presented with syncope and gross (red) blood per rectum. History was significant for Apixaban for atrial fibrillation, and NSAID use. My questions to the team were: what are the possible etiologies of the bleeding; and which among are most likely? After the diagnostic possibilities were enumerated, I offered my wager: a $10 stake that it is peptic ulcer disease (PUD) at 4:1 odds. If the patient has PUD, a taker of my wager owes me $10. If the patient does not have PUD, I owe the taker the $10 stake multiplied by the odds: $40.

There are several pedagogical and practical reasons I engage my team in these games, most of them pedagogical. First, making a wager about clinical outcomes gives you skin in the game. It is remarkable how, when I assign a stake and odds, the interest level goes up and my would-be wager-takers become carefully attuned to the differential diagnosis and other pieces of data that until then had been glossed over or wholly neglected. Recently, when making a wager about results of a pending echocardiogram, the resident did a much deeper chart dive than had previously been done and found a remote echo which bore on the probabilities for the current wager. I had to reassign the odds in light of this new information. Related to this deeper search for available information about the case, there can be a search for information about the diseases under consideration. It would be folly to try to make a bet about the probability of a disease that you understand incompletely, or not at all. Skin in the game will likely lead you to www.uptodate.com where you can ascertain knowledge about the considered diseases to apply it to the wager.

Bayes' Theorem Explained, No Math Required

I was asked by a medical student to explain Bayes' Theorem.  This blog is about lack of common sense in medicine, so it follows that education about first principles will contribute to uncommon sense, and I will oblige.

Bayes' theorem is simply a long or holistic way of looking at the world, one which is more in keeping with reality than the competing frequentist approach.  A Bayesian (a person who subscribes to the logic of Bayes' Theorem) looks at the totality of the data, whereas a frequentist is concerned with just a specific slice of the data such as a test or a discrete dataset.  Frequentist hypothesis testing is where we get P-values from.  Frequentists are concerned with just the data from the current study.  Bayesians are concerned with the totality of the data, and they do meta-analyses, combining data from as many sources as they can.  (But alas they are still reluctant frequentists, because they insist on combining only frequentist datasets, and shun attempts to incorporate more amorphous data such as "what is the likelihood of something like this based on common sense?")

Consider a trial of orange juice (OJ) for the treatment of sepsis.  Suppose that 300 patients are enrolled and orange juice reduces sepsis mortality from 50% to 20% with P<0.001.  The frequentist says "if the null hypothesis is true and there is no effect of orange juice in sepsis, the probability of finding a difference as great or greater than what was found between orange juice and placebo is less than 0.001; thus we reject the null hypothesis."  The frequentist, on the basis of this trial, believes that orange juice is a thaumaturgical cure for sepsis.  But the frequentist is wrong.

The Medical History as an Exposure Narrative: A Didactic for Medical Students and Young Physicians

I just received an email from a medical student on the other side of the pond asking for my advice for junior doctors for learning the practice of medicine.  I will oblige.  When students are taught how to take a medical history, they are taught a rote sequence and its components, but are not taught what the point really is, why we ask certain questions, how we string the answers together, which components need more or less emphasis in a given case.  Here I will present a framework for that understanding, which may make history taking more meaningful and useful for those learning and refining it.

What we are really trying to do with history taking is to make a narrative of the patient's exposures in his or her environment.  This exposure narrative allows us to use Bayes' Theorem to determine the most likely causes for a given chief complaint.  Bayes Theorem should be reviewed for its own sake in order to understand its use here and elsewhere, but simply understanding that the base rate of a disease in a certain population is the "prior probability" of that disease in a patient will suffice for now.  So, if I asked you what mammal you are likely to see on your hike in the Rocky Mountains, you will list squirrel and deer and elk before mountain lion and badger.  It's just common to see deer and elk there.  In the jungle, or the desert, the answers would be different - because different animals have different probabilities in different environments.  Likewise, when a 70-year-old comes with pain in the joints, osteo- and rheumatoid arthritis are more likely than lupus and juvenile rheumatoid arthritis, which would have higher probabilities in younger patients.  Thus, age is an exposure, perhaps one of the most important ones, and this is why a student's presentation often begins with something like "Mr. Jones is a 78-year-old man..."  (It is also why I frequently interrupt physicians who call me to admit a patient or consult on one, because I can't begin to order the probabilities until I know the age of the patient, which they frequently omit because of indolence.)  The older person has been exposed to wear and tear on the body for a longer time, and this figures prominently in the probabilities of the diseases that s/he is likely to have.