Epistemic Arrogance. Case Records of the Massachusetts General Hospital. Diagnosis is Stochastic, NOT Deterministic

This post was spurred by my reading the July 24^th NEJM case records, “A Man with Cough, Dyspnea, and Hypoxemia.”  When I was still using twitter, I mused that the discussants have too high a rate of getting the right diagnosis. This implies that there is no stochastic or aleatory uncertainty in diagnosis; that if you have the requisite information and reasoning process, you can connect the dots, every single time, to the final diagnosis. In this view, there is no randomness to diagnosis, no inherent and irreducible probabilistic (or stochastic, or aleatory) uncertainty. Of course, this is pure codswallop: stochastic uncertainty is what makes diagnosis so hard.

You cannot say that the result in any case was not possible with the information presented, but with a population of cases you can make inferences about whether the percentage of correct discussant diagnoses is improbable. The fact that the discussant gets the right diagnosis almost 100% of the time (I have tabulated the % correct for over 10 years of these cases – it rounds to 100%) in the CRMGH (Case Records of the Massachusetts General Hospital) is like finding too little variability in a research study (or  studies; or Bernie Madoff’s annual returns) and concluding that the results (returns) were manipulated. We use our well-justified first principles assumption of stochasticity, find that there is not enough variability, and conclude that something is being manipulated to remove the aleatory uncertainty. (I use stochastic, aleatory, and probabilistic herein as interchangeable adjectives for uncertainty.) And so must it be in the CRMGH. How the aleatory uncertainty is being removed is anybody’s guess and any insider’s knowledge, I suppose. Possibilities include both implicit and explicit cues. An implicit cue may be the selection of, say, a specialist in heritable cardiomyopathies as the discussant in a case where the final diagnosis is a genetic cardiomyopathy. Explicit cues may be that somebody tells the discussant the final diagnosis at the outset or tells them to revise their discussion if they get it wrong in draft. Hospitals are not closed systems either. People hear if there was a case of Creutzfeld-Jacob diagnosed recently, and the director of the microbiology lab knows if a particular rare species was recently cultured. Tales of rare diagnoses spread, as they are the currency of clinical acumen. I can’t say that any of these mechanisms -- from subtle queueing to explicit cheating -- are occurring. I only know that the near 100% hit rate means that randomness has been removed from an inherently random process.

The case that prompted this post (linked above) was that of a 75-year-old man with COPD who had received multiple courses of corticosteroids for COPD and possible ABPA and is thusly immunosuppressed, has an extensive travel history including Asia, and central America recently (where, exactly, we are not told) and has multifocal consolidative opacities including one with cavitation. (Whether there is lymphadenopathy we are neither told nor shown.) Despite the epistemic uncertainty posed by incomplete information, the perspicacious reader of the report quickly determines that infection, especially an opportunistic one, is likely to be the diagnosis. But which one? I quickly formed the following differential:

Garden variety infection with an unusual presentation (always the best bet, but one must allow for the fact that these cases are selected against)

Mycobacterial disease (primary TB or atypicals)

Fungal disease, especially (para)coccidiodomycosis because of the onset of symptoms during travel to Central America, despite it being unclear when/if the patient had stopped voriconazole for possible ABPA.

Nocardia, because it is ubiquitous and tends to give golf-ball sized cavitary lesions like that in the RUL; however, large consolidative opacities like the LLL are rarer.

Possibly malignancy or an inflammatory process (e.g., Wegener’s, but we’re not told about a urinalysis)

Spoiler alert: the discussant identified Nocardia as the diagnosis, and, of course, she was correct. How did she do it? By going through all the differentials above and a couple of others, and then excluding them, because the present case does not have the “typical features”. Viz:

As for Paracoccidioidomycosis (which I have never seen; regular cocci is more common, even in Central America, a useful epidemiological fact that the discussant neglects): “but large mass lesions are not typical of this infection”

As for parasites: “However, this patient did not have the gastrointestinal symptoms….that are typical of parasitic disease”

As for mycobacterial infection: “However, the large lower-lobe mass lesions that were seen in this patient would be atypical”

But when we get to nocardia, the discussant admits that lobar consolidations are atypical for this organism, too, but they rarely occur. With this, we have a tacit admission that the “typicality” criterion previously relied upon is not useful in this case. Despite that, and the lack of typical CNS dissemination, the discussant manages to pin down nocardia as the final diagnosis.

Leaving aside the fact that what counts as “typical” is not defined at all, and that the likelihood ratios for typical and atypical are unknown (and probably low), it certainly appears that “typicality” was used to exclude diseases selectively. Which leaves me at a loss as to how the described “differential diagnosis” process (if it is fair to call it that at all), can be applied to other cases – sometimes you exclude diseases because they’re atypical, other times you ignore typicality. It smacks more of an exercise in motivated reasoning (or foreknowledge of the diagnosis), than a process that can be prospectively applied in future cases.

Of course, the twist is that the patient also had M. abcessus, though it is unclear if it was a false positive, an incidentaloma, a co-infection, or a commensal. But if you read this case and think that the data pointed deterministically to nocardiosis, somebody is pulling the wool over your eyes.

When I teach residents and students, I stress how hard diagnosis is because of stochastic uncertainty, and the need for “epistemic humility”. The NEJM is doing a disservice to learners by suggesting, week after week, that diagnosis is a deterministic process. Decidedly, it is not.