Friday, July 5, 2019

The Truth Doesn't Always Need A Test: Thresholds for Medical Decisions

Jason Carr, MD didn't need a test to know what this is
Kassirer (and Pauker) got the idea for the Threshold Approach to Medical Decisions in part (as he tells it in his memoir Unanticipated Outcomes) because he had observed a pattern of irrational behavior among nephrologists (he was one) in the 1960s.  Evaluating a patient with nephrotic syndrome and knowing the diagnosis was very likely to be membranous glomerulonephritis, most nephrologists still insisted on a biopsy prior to starting corticosteroids.  Pauker and Kassirer realized that we never get to 100% certainty in medicine and that tests only modify the pre-test probability of disease.  They reasoned that there ought to be a threshold of pre-test probability of disease that is high enough to justify treatment without testing.  That is, sometimes, the pre-test probability of disease is so high as that it obviates testing, and we can just skip to treatment and avoid risky and invasive testing like a kidney biopsy.  Only if the disease fails to respond to therapy (thereby reducing the clinical pre-test probability below the treatment threshold) should we resort to the biopsy.

Almost 40 years after the publication of their article, we still are not consistently following the axioms of the threshold approach.  Here are several recent examples:
A 20-something black woman presents with cough, weight loss, massive hilar and mediastinal lymphadenopathy and interstitial parenchymal changes.  She is scheduled for a bronchoscopy and EBUS to confirm the diagnosis of sarcoidosis prior to initiating corticosteroids.
The probability of sarcoidosis in this case is in excess of 90%, and a response to treatment with corticosteroids will drive the probability to near certainty, just as a bronchoscopy showing non-caseating granulomas will.  A negative EBUS, with approximately 80% sensitivity for sarcoidosis, will not "rule out" sarcoidosis - using the calculator on the sidebar of this blog you can plug in that sensitivity and, say 99% specificity, and find that the posterior probability of sarcoidosis with a negative EBUS is still 65%!  Remember from the last post that "the test is not the truth."

So then why do we almost universally "confirm" the diagnosis with bronchoscopy and EBUS?  Firstly because we do not generally adhere to the axioms of rational decision making; second because EBUS is remunerative; third because we ignore the costs and risks of the invasive test (I'm still salty 17 years later after I was forced to do transbronchial biopsies in a lady with obvious sarcoidosis and caused her to have a pneumothorax in fellowship);  fourth, because we are reluctant to estimate clinical pre-test probabilities; lastly, because we inflate the risk of missing something.  Let's focus on the last one for a moment.

Lymphoma.  Everybody is worried they're going to miss a lymphoma.  Well, if the case were less classic for sarcoidosis, or had features worrisome for lymphoma, that means the pre-test probability of sarcoidosis may not be above the treatment threshold, in which case, saddle up the bronchy donkey!  Let's say that in 100 cases like our case, the probability of lymphoma is, for the sake of argument, 5% and that the probability of a dramatic response of lymphoma to corticosteroids is 10%.  Assuming independence, the probability of finding ourselves in the situation where we missed a lymphoma and it responded dramatically to steroids thus completely mimicking sarcoidosis is 0.5%.  That's going to be on the high end, because I intentionally inflated those probabilities.  (CNS lymphoma is a special case where there can be dramatic shrinkage with steroids alone.)

But there's more.  Does the bronchoscopy completely exclude lymphoma?  Certainly not!  It has neither 100% sensitivity for lymphoma nor 100% specificity for another diagnosis.  It does, however, provide a false sense of security for anybody who thinks the test is the truth.

The bottom line is that at a certain threshold of pre-test probability, the test has diminishing marginal diagnostic utility, so its value compared to its risks and costs is lower.  (The model for determining testing and treatment thresholds in the 1980 paper is incomplete because it does not consider costs of testing, and it treats all adverse outcomes as equal.  This is necessary to simplify the math, which is cumbersome enough even with the simplification.  I favor a more qualitative threshold approach, as outlined above.)
A 50ish female immunosuppressed for a solid organ transplant developed dyspnea on exertion and dry cough 1 month ago coincident with a tapering of corticosteroids.  There are now bilateral interstitial and ground glass opacities, a new oxygen requirement, an LDH level of 650.  A "bronch" is requested for possible PJP pneumonia.
The probability of PJP in this case is on the order of 80-90% (prior odds 4:1 to 9:1 for those who, like I, prefer the odds formulation of Bayes Theorem).  This is probably approaching the treatment threshold already, but since Bactrim (TMP/SMX) is so toxic (a cost of treatment that we don't want very many to bear if they don't actually have PJP), we may wish for it to be 95% or higher.  That's where 1,3-beta-D-glucan, with its sensitivity of 95% and specificity of 82% for PJP comes in.  With a pre-test probability of 80% and a positive 1,3-beta-D result, we make the 95% treatment threshold (plug some numbers into the calculator and see for yourself).  Luckily, the sputum PCR for PJP was also positive, so we didn't even need to campaign to convince folks that the clinical pre-test probability and the positive 1,3-beta-D results were sufficient to clinch the diagnosis and begin treatment.
A 40ish man presents with necrotizing pneumonia, severe pleuritic chest pain, and a parapneumonic effusion.  Should we do a thoracentesis, send fluid for pH and then make a decision about tube thoracostomy drainage, or just put in the tube?
Here again, the pre-test probability of a complicated parapneumonic effusion is quite high, on the order of 90%.  This probability is arguably above the treatment threshold for a chest tube, especially since the pH criterion itself is a judgment call with different experts suggesting different threshold values.  Nine times out of ten, this will save the patient from a thoracentesis in addition to a chest tube.  I further posit that the treatment threshold for chest tube insertion has gotten lower in the past 20 years.  It used to be that if your pH were below 7.20, you got a big, painful, surgical chest tube.  That is a cost that you don't want anybody to bear unless you're almost certain they need it.  Nowadays, the percutaneous Seldinger inserted tubes are very benign and well tolerated, and not much bigger than an in and out thoracentesis drainage catheter.  When the costs of treatment in the small proportion of patients who don't "need" it get lower, the treatment threshold gets lower in parallel.

By the way, the pH was 7.07.  We often send them after we've already put in the tube as feedback to monitor ourselves.  If our rate of tube insertion with pH >7.20 goes above 10%, we learn that we need to adjust our pre-test probabilities of "complicated parapneumonic effusion".


1 comment:

  1. The patient was intubated after flexeril overdose for "airway protection." Several hours later he awakens and is furiously pulling at the restraints. Is a SAT/SBT indicated prior to extubation?

    ReplyDelete

Note: Only a member of this blog may post a comment.