This post was spurred by my reading the July 24th NEJM case records, “A Man with Cough, Dyspnea, and Hypoxemia.” When I was still using twitter, I mused that the discussants have too high a rate of getting the right diagnosis. This implies that there is no stochastic or aleatory uncertainty in diagnosis; that if you have the requisite information and reasoning process, you can connect the dots, every single time, to the final diagnosis. In this view, there is no randomness to diagnosis, no inherent and irreducible probabilistic (or stochastic, or aleatory) uncertainty. Of course, this is pure codswallop: stochastic uncertainty is what makes diagnosis so hard.
You cannot say that the result in any case was not possible with the information presented, but with a population of cases you can make inferences about whether the percentage of correct discussant diagnoses is improbable. The fact that the discussant gets the right diagnosis almost 100% of the time (I have tabulated the % correct for over 10 years of these cases – it rounds to 100%) in the CRMGH (Case Records of the Massachusetts General Hospital) is like finding too little variability in a research study (or studies; or Bernie Madoff’s annual returns) and concluding that the results (returns) were manipulated. We use our well-justified first principles assumption of stochasticity, find that there is not enough variability, and conclude that something is being manipulated to remove the aleatory uncertainty. (I use stochastic, aleatory, and probabilistic herein as interchangeable adjectives for uncertainty.) And so must it be in the CRMGH. How the aleatory uncertainty is being removed is anybody’s guess and any insider’s knowledge, I suppose. Possibilities include both implicit and explicit cues. An implicit cue may be the selection of, say, a specialist in heritable cardiomyopathies as the discussant in a case where the final diagnosis is a genetic cardiomyopathy. Explicit cues may be that somebody tells the discussant the final diagnosis at the outset or tells them to revise their discussion if they get it wrong in draft. Hospitals are not closed systems either. People hear if there was a case of Creutzfeld-Jacob diagnosed recently, and the director of the microbiology lab knows if a particular rare species was recently cultured. Tales of rare diagnoses spread, as they are the currency of clinical acumen. I can’t say that any of these mechanisms -- from subtle queueing to explicit cheating -- are occurring. I only know that the near 100% hit rate means that randomness has been removed from an inherently random process.
