The Test is Not the Truth: One Week in the Lonely Life of a Bayesian Clinician

DAH from GPA or CPE/ESRD?

If there is one thing you should remember about clinical decision making it is this:  the test is not the truth.  A diagnostic test raises or decreases the prior or pre-test probability (PTP) of the disease under consideration.  The amount of increase or decrease in probability with a positive or negative test depends on the starting probability and the likelihood ratio of the test.  (LR+ = sensitivity/1-specificity; LR- = 1-sensitivity/specificity).  If we don't attend to the PTP of disease, serious diagnostic errors and therapeutic misadventures may result.  This is especially true when a low PTP disease is diagnosed on the basis of a test with poor sensitivity and specificity (and a LR not much greater than 1 or 2 or even 4 or 5).  Several examples of this came up a while back.

A woman presented with thunderclap headache and had recurrent seizures during initial evaluation.  A differential diagnosis was formulated and it included PRES (posterior reversible encephalopathy syndrome) with a PTP of about 20%.  Subarachnoid hemorrhage was excluded with CT and LP and the PTP of PRES rose to about 40% (since it occupied some of the probability space previously occupied by SAH once the latter was excluded.)  The subsequent MRI images were consistent with PRES.  Nonetheless, a vascular MRI was ordered to "exclude the possibility of cerebral vasculitis".  The problems are twofold.  First, the probability of PRES is now on the order of 70% if the sensitivity and specificity of MRI are on the order of 80%, and it is 85% if sensitivity and specificity are each 90%.  (Go ahead and plug some numbers into the calculator on the sidebar of the blog.)  This probability meets or exceeds the probability threshold to both consider the diagnosis made, and to take action based on it.  In this case inaction and supportive care are indicated.  Even if a vascular MRI were consistent with cerebral vasculitis, which has a PTP an order of magnitude or more less than PRES, the diagnosis is still PRES.  The truth is not in the test, the truth is in the rationally considered diagnostic process of which the test is one part.

Limits of the Possible: Clinical Reasoning of a Harrowing Extubation

"The only way of discovering the limits of the possible is to venture a little way past them into the impossible."  -  Clark's Second Law

In prior posts here and on the Medical Evidence Blog (here, here, here, and here), I have outlined my position that the only way you can really know if a patient can breathe on their own is to let them try - a "trial of extubation".  Prediction equations get you published, but their signal to noise ratio is often poor and ignored, to patients' peril.  Indeed the reason I'm obsessed with extubation is because I think being intubated unnecessarily is one of the worst things a patient can endure, and the best thing I can do as an intensivist is identify the earliest moment when a patient can breathe on his own and extubate him.

I faced a very harrowing extubation decision recently, and I admitted to the medical students that it was the most nail-biting of my career.  But I think analyzing it, both before and after the fact can be very instructive.

Medical Decision Making as a "Patient": Pregnancy Leads to A Trip Down The Rabbit Hole - A Personal Story

My wife is pregnant.  Wanting to be a supportive spouse, I attended the first prenatal visit to see one member of her team of midwives.  (Being a "minimalist" I was, like my wife, fond of the idea of not unnecessarily "medicalizing" the [usually] natural act of labor and birth.)  I realized during that first visit that understanding the intricacies of medical decision making can be a double-edged sword when dealing with practitioners, especially outside of one's specialty.  If ignorance is bliss, 'tis folly to be wise, it is said.  I've come to wonder which is better for you when you get entangled in US healthcare, wisdom or bliss.

During the first visit, we were offered, with an air of agnosticism, a referral for genetic counseling +/- non-invasive prenatal testing (NIPT).  "How accurate is it," I naturally inquired, trying to avoid technical terms such as sensitivity and specificity.  "Something like 99%" came the reply.  So we were given the referral.  But I quickly realized that this was a classic problem of base rates.  The likelihood of a chromosomal abnormality is so low given my wife's age, that even extremely high sensitivities and specificities are inadequate to guide our decision - that is, the test is rendered practically useless because of the low base rates in our case.  And this despite the fact that the sensitivities and specificities of prenatal blood testing are inflated by the way they were derived.  But think of the decision we would have faced had we blindly proceeded with testing without this consideration - given the low base rate, the posterior probability of a chromosomal abnormality such as Down's Syndrome given a "positive" test result would be around 33%.  How would we act on this information?  Is that threshold high enough that we would consider an elective abortion (if we were morally disposed towards that as an option)?  Or would we ignore the information and proceed to term?  And if we were not ethically accepting of elective abortion as a possibility, what other remedy would we have that would justify the information from the testing?  Why would we talk about getting prenatal genetic testing before talking about the choices we may have to face after we receive the results?  Why would not a discussion of remedies, specifically abortion, precede consideration of the testing?  How many couples dive into the rabbit hole only to wonder how they got there and how they can get out?  In this case, we decided that ignorance was indeed bliss, and deferred NIPT.

At that same visit, blood was ordered to be drawn.  I had difficulty understanding why you would need to draw blood from a perfectly healthy woman at 12 weeks gestation.  Blood types and anemia and all that I guessed.  But I was particularly caught by the thyroid testing.  Why are we screening an asymptomatic woman for thyroid disease?  Is that justified by the prior probabilities?  It takes only a google search to learn that ACOG (the American College of Obstetrics and Gynecology) and an endocrine society do not recommend universal testing.  But my questioning why we were doing this was off-putting and frankly unanswerable for the midwife - she was just following the usual routine, whatever her supervisors and mentors had told her to do, without understanding....well without understanding any of this Bayesian mumbo jumbo that I was hinting at.  Alas, thyroid testing, like NIPT, was deferred.  But not for long.

A Poor Predictor is Worse than No Predictor: On the Superiority of Empiricism in Some Medical Decisions

John Locke, empiricist.

The moral of this story is that much maligned empiricism is sometimes (often?) both the only thing to guide you and also the best thing to guide you.

I recently received a call (at an odd hour on the weekend) from an otolaryngologist (ENT) regarding a patient from whom she had drained a large submandibular abcess.  She was calling to tell me that she planned to leave the patient intubated in the ICU overnight and she wanted "help with ventilator management" (which of course the patient does not need - he can be managed with an endotracheal tube not connected to any mechanical ventilator).  The patient did not have airway compromise or concerns thereof prior to surgery, but, she said, there was swelling noted after the case that (for her) raised concerns about the patency of the patient's airway if the endotracheal tube were to be removed.

(There is a second moral to this story: far too often, patients such as this are left intubated post-operatively not for their own safety, but rather for the convenience of surgeons and anesthesiologists who do not wish to spend the extra time awakening them from anesthesia and observing them carefully in the post-anesthesia care unit.  It is far easier to not fret over the depth of anesthesia, atelectasis, oxygen levels, fluid status, and leave the patient intubated and send them to the ICU and let somebody else sort it out.  If I had a family member undergo a relatively routine, even if urgent or emergent operation at an odd hour [holidays, weekends, after hours] and they were sent to the ICU post-op for no apparently good reason, there would be hell to pay.  Note also that for the surgeon and anesthesiologist to save an hour of their time, another physician has to drive to the hospital to take over for them spending hours of his time, and also often a nurse must be called in from home to accommodate the unexpected post-op admission [as was the case here].  The sheer arrogance and egocnetricity of this is mind-boggling.  But I digress.)

Back to the story.  I naturally inquired as to what criteria we would use the next day to determine if the patient's oropharyngeal swelling had abated sufficiently such that we could safely extubate him.  The ENT replied that she would scope (endoscopy) the patient again in the morning and if the swelling had decreased we could proceed with extubation (removing the endotracheal tube).  Well and good.  Or is it?

Bayes' Theorem Explained, No Math Required

I was asked by a medical student to explain Bayes' Theorem.  This blog is about lack of common sense in medicine, so it follows that education about first principles will contribute to uncommon sense, and I will oblige.

Bayes' theorem is simply a long or holistic way of looking at the world, one which is more in keeping with reality than the competing frequentist approach.  A Bayesian (a person who subscribes to the logic of Bayes' Theorem) looks at the totality of the data, whereas a frequentist is concerned with just a specific slice of the data such as a test or a discrete dataset.  Frequentist hypothesis testing is where we get P-values from.  Frequentists are concerned with just the data from the current study.  Bayesians are concerned with the totality of the data, and they do meta-analyses, combining data from as many sources as they can.  (But alas they are still reluctant frequentists, because they insist on combining only frequentist datasets, and shun attempts to incorporate more amorphous data such as "what is the likelihood of something like this based on common sense?")

Consider a trial of orange juice (OJ) for the treatment of sepsis.  Suppose that 300 patients are enrolled and orange juice reduces sepsis mortality from 50% to 20% with P<0.001.  The frequentist says "if the null hypothesis is true and there is no effect of orange juice in sepsis, the probability of finding a difference as great or greater than what was found between orange juice and placebo is less than 0.001; thus we reject the null hypothesis."  The frequentist, on the basis of this trial, believes that orange juice is a thaumaturgical cure for sepsis.  But the frequentist is wrong.