Pitfalls of Protocols: Pushing the Limits of Extubation

A recent post described extubating an asthmatic patient with very bad weaning parameters, and I promised to provide a followup telling whether he "flew" or not.

He flew.

It was a nail-biting experience and for the first hour it was unclear if he was going to make it.  His respiratory rate settled down into the teens which was reassuring, but he did not gain lucidity for quite a while and was intermittently midly "combative" and uncooperative.  He was on HFNC with oxygen saturations in the high 80s and low 90s, and he remained tachycardic and was wheezing.  His wife and the nurse were continuously in the room reassuring him, as were several doctors during the first hour or two.  During the course of several hours, he was able to be weaned to simple face mask and then nasal cannula and the next day he was discharged from the ICU.

If we had followed some sort of "weaning protocol" with blind faith, I don't know how long it would have been before he would have "passed" the protocol's tests and been extubated.  At some point, somebody would have said, "Hey, we better stray here, or we'll never get him extubated."



This case and many others described on this blog highlight the pitfalls of protocols, which are designed to approach best performance for "typical" patients cared for by non-experts.  (That's a furtive way of saying "protocols are not for me.")  For example, in this study at Johns Hopkins from 15 years ago, protocol-driven weaning was not shown to be superior (and the numbers were almost identical in each group) for extubation success and time on vent when a protocol group was compared to a usual care group.

Hopefully you caught me there.  That would be a classic misuse of logic and statistics.  Absence of evidence is not evidence of absence.  What we want to know is the power of the study to find a difference of a certain size, or, even better the 95% confidence interval of the result, so we can determine the range of values that are compatible with the result.  The authors state:

The sample size provided 82% power (assuming two-sided type I error < 0.05) to detect a 1-day difference in mechanical ventilation duration.

What this means is a conditional probabilistic proposition:  "Given that there is in truth a difference of one day or more between the two groups, our study with a sample size of 150 patients in each group has an 82% probability of finding the difference at the 0.05 significance level."

Note that this is not what we want to know.  We want to know if there is a difference, not what the probability of finding it is.  We didn't find it!  Is it there or not?  It's like saying "Given that there are Elk in the Wasatch mountain range, there is an 80% chance that we will find one with our search strategy."  If we don't find any elk, we still don't know if there are elk, we only know that we had an 80% chance of finding them. 

The authors didn't report the standard deviation (SD) for the primary outcome, the duration of mechanical ventilation, so we can't calculate a 95% confidence interval for the difference.  (An aside:  I know the authors of this trial and I'm surprised they didn't report the 95% CI for the difference.  I could estimate the SD from their Figure 2, if the data were normally distributed, but the distribution is highly skewed, so I don't want to commit a foundational error there.) A proportional outcome, the percentage successfully extubated was about 75% in each group and we don't need a SD for a proportion.  The 95% CI for the difference in these proportions with 150 patients in each group is approximately -10% to +10%.  Whether this is good enough for us, to conclude that a difference in successful weaning of 10% in either direction is negligible, I don't know. 

In any case, I submit that regardless of these data, protocols are there to get patients extubated, not keep them intubated.  Perhaps patients would be best served if we believe it when they pass an SBT and pause for second thoughts when they don't.