Numeracy in Medicine: Clinical Wagers to Estimate Probability in Diagnosis

As has become my custom, I offered a wager on rounds the other day. An nonagenarian had gastrointestinal hemorrhage. He presented with syncope and gross (red) blood per rectum. History was significant for Apixaban for atrial fibrillation, and NSAID use. My questions to the team were: what are the possible etiologies of the bleeding; and which among are most likely? After the diagnostic possibilities were enumerated, I offered my wager: a $10 stake that it is peptic ulcer disease (PUD) at 4:1 odds. If the patient has PUD, a taker of my wager owes me $10. If the patient does not have PUD, I owe the taker the $10 stake multiplied by the odds: $40.

There are several pedagogical and practical reasons I engage my team in these games, most of them pedagogical. First, making a wager about clinical outcomes gives you skin in the game. It is remarkable how, when I assign a stake and odds, the interest level goes up and my would-be wager-takers become carefully attuned to the differential diagnosis and other pieces of data that until then had been glossed over or wholly neglected. Recently, when making a wager about results of a pending echocardiogram, the resident did a much deeper chart dive than had previously been done and found a remote echo which bore on the probabilities for the current wager. I had to reassign the odds in light of this new information. Related to this deeper search for available information about the case, there can be a search for information about the diseases under consideration. It would be folly to try to make a bet about the probability of a disease that you understand incompletely, or not at all. Skin in the game will likely lead you to www.uptodate.com where you can ascertain knowledge about the considered diseases to apply it to the wager.

Second, if you're going to make a valid prediction about something, you have to be specific. It's not enough (but it's a good start) to list the entire differential diagnosis and pat yourself on the back when the final diagnosis is among your long list of possibilities. Some of those possibilities are far more likely than others, and unless the probabilities are considered you won't know which ones to test for or empirically treat for at the first. If, as some unfortunately do, you test for pulmonary embolism on a large percentage of your patients because you fail to consider probability, you're wasting valuable resources. Similarly, if you give vancomycin and meropenem  to every patient regardless of the probability of resistant organisms, you're abetting antibiotic resistance. All of our decisions hinge of probability. (The threshold approach has been discussed elsewhere and is beyond the scope of this post.)

Third, assigning probabilities helps you gain feedback on your decisions. If you name the entire diagnostic possibility list without transforming it into a differentiated diagnosis list - by differentiating among the possibilities, and assigning relative probabilities to them - you can never be wrong unless your list fails to include the final diagnosis. You will always look back and say, "yeah I knew the low platelets were from [sepsis, ITP, TTP, etc]" and your learning will be fatally handicapped by hindsight bias. If, however, you place a wager (whether a financial one or a psychological one) with decent odds (say 2:1) on TTP and the final diagnosis turns out to be sepsis, you will be forced to reconsider why you betted on a rare rather than a common disease. Further, you may be prompted to go back - if you had not already - and ask some questions about rigors to determine why you gave short shrift to sepsis; or review the laboratory findings of TTP.

Fourth, wagers militate against overconfidence. By custom, I arrange the wager so that the person who is most confident in an outcome is the maker or the wager at odds equal to or in excess of 1:1 (even odds or an "even bet"). That arrangement assures that the person most confident (and usually quite overconfident) in the outcome pays the price for their hubris in the case of a lost wager (where the payout is the odds multiplied by the stake, say $10). The effect is often that, when considering the steep payout if the wager goes against them, the maker reconsiders the high probability they originally assigned to the outcome and revises the odds accordingly. As an added benefit, some attention gets refocused on the diagnostic possibilities that were being short-changed when too high of odds were assigned to the favored diagnosis. When I began doing this about 15 years ago, I had a habit of assigning 9:1 odds, but I got burned too many times and the resultant circumspection has helped cure me (somewhat) of my overconfidence.

In sum, we can actually estimate probability in medicine, using epidemiological (and test) data and clinical experience and preferably a combination. Assigning odds to clinical wagers creates leveraged payoffs that militate against overconfidence. In the case of the elderly patient with GIB, it was indeed PUD, but I didn't make the taker of my wager pay, as a courtesy. If I had lost the wager, and had to pay up $40, I would have been forced to consider, by paying a pound of flesh with my skin in the game, whether my confidence in PUD had been too high. In a future post, we can discuss how Brier Scores (which by their nature - they are a mean squared error method - also penalize overconfidence) could likewise be used to monitor probability prediction accuracy and overconfidence. In this case my Brier Score would have been 4, with lower scores being better. If I had given 9:1 odds, corresponding to 90% probability, my Brier score would have been 1 which is pretty spectacular; however, if I had lost at 9:1 my payout to the taker of the wager would have been $90 rather than $40, a steep penalty to get my score from 4 to 1.