Countless Hours, Part 3: Uncalibrated Interns and Immediate and Accurate Feedback

Immediate, accurate feedback begets calibration

In this third and final installment of Countless Hours:  How to Become a Stellar Student and An Incredible Intern, I will discuss the role of immediate and accurate feedback for the refinement of a skill, prediction, or prognostication to expert levels.

Imagine you are learning to play golf, but you can't see where your balls are going - it would be very difficult, without any feedback to learn to modify your swing to improve your game.  Similarly, if the feedback you received were from an observer with poor vision, and it was not accurate, you would be trying to calibrate your swing to unreliable information and your game would not improve insomuch as the feedback was inaccurate.  Finally, if you did not receive the feedback on your swings until days later, it would be difficult to analyze it and adapt your game to it, compared with iterative feedback incorporated after each swing.

The same principles apply to learning the practice of medicine.  One of the reasons that the case study books mentioned in the previous post are so instructive is that they provide immediate, and accurate feedback - you get to know if your diagnosis was correct immediately after rendering it, and this feedback, from the experts who wrote the book (often with formal or informal peer review and editing), is presumably as accurate as you can hope for.  Thus, case based practice is a very very effective way to become an expert.

Then there is the "hands on" work you do on the wards and in clinic.  Here, feedback is, on average, less immediate, and less accurate, and this is one of the ways your learning in "real life" scenarios is compromised - but there are several things you can do about it to maximize the immediacy and accuracy of feedback in these environments.


Immediate feedback is one of the best reasons I can think of to justify long stretches of being "on call" during your training, and indeed discussion of it was curiously absent during the debates about resident work hours.  When you are on call for 36 hours straight, you get to see in real time the effects of the decisions you have made regarding a newly admitted patient, and you get to see the test results as they roll in.  Compared to admitting that patient, and then coming back 12 or 24 hours later, your learning will be greatly enhanced because the feedback is immediate, and also, because you are there personally witnessing it, it will have more emotional content and thus be more memorable.  (I am reminded of the woman on 5 West during my internship with phegmasia cerulea dolens to whom I gave a whole bunch of opioids while on call one night because I was convinced we were not adequately treating her pain and she stopped breathing [almost] and a code was called.  Being the intern on call, and seeing that the code was "my fault" certainly made that a memorable experience, far more so than the same experience reported back to me the next day.)   If you are not on call to receive feedback, check the labs and tests you ordered from home, or call to inquire so you can get the feedback on your decisions as promptly as possible.  Being around to see the results of your decisions, good and bad, is an invaluable learning experience, especially if you have the fortitude and insight to criticize your own decisions.

Oh, yes, the fortitude and insight to criticize your own decisions.  To become a calibrated expert decision maker, you have to receive and internalize feedback, that of others and that from yourself.  But this uncovers a dilemma and a conflict for most people:  they have as a goal to become better (by internalizing feedback) but they also have as a goal to believe that they are already good and that they got it right the first time.  When you "solve a case" and commit to your answer or your diagnosis (often publicly, which puts the fragile ego in a precarious position), you want to be right.  This want can lead to failure to accept or internalize feedback, or worse yet, to fail to incorporate additional incoming information and to modify your diagnosis before the final diagnosis is known.  This kind of psychology is very prevalent, and is the enemy of decision making and calibration.  You MUST remain open to your errors and to try to understand how you made them.  Your ego must take a back seat.  And many many times, you must be the judge and jury of your own decisions, and you must be very honest with yourself even when that is very hard to do.  I am not encouraging your to self-flagellate each night like a Jesuit Priest, just to be introspective about your mistakes, and to always consider if you may be missing something or making a mistake.

Some specialties by their nature are isolated from feedback and this harms their calibration enormously.  Consider the ER - they admit a patient with chest pain for "angina" and 24 hours later during a cardiac catheterization, pulmonary embolism is diagnosed.  If the ER physician who made the angnia/myocardial infarction call does not follow up on this case, s/he will never get the feedback s/he needs to calibrate future decisions.  Thus, in some circumstances it may be necessary to actively seek out feedback that the environment either does not make evident, or actually conceals from you.

Another problem with feedback is inaccuracy.  If the feedback you receive is totally random, it does you no good.  And here's a dirty little secret:  your intern in July whom you follow around as a med 3 - how much better than random do you think his feedback is?  Suppose that while admitting her you listen to Mrs. Jones' heart and you hear a systolic murmur.  You think it is a holosystolic murmur, and it does not vary with extrasystoles, and you decide it's probably MR based on these facts and perhaps some others.  You tell your intern about it and he says "no that's just a flow murmur, lots of patients have those."  I can tell you right now that that feedback is probably useless.  Even if Eat-My-Dust-Eddie your intern listened to the murmur carefully, he probably is himself uncalibrated and thus nearly useless as a feedback provider for you.  Sorry, interns everywhere, but it's mostly true.  (Perhaps the worst thing you can do is to ask him and then argue with him.  Worse still is to stick to your guns when you present to your attending, and ultimately be right, effectively "showing up the intern".  Ask me how I know.  And ask yourself why your intern evaluates you at all on your medical school rotations given that s/he is probably uncalibrated at that stage in their training.)

I solved this problem beginning in the third year of medical school in two ways.  First, I quit asking the house staff what they thought about this and that murmur (or I at least took it with a grain of salt) because I learned early on that they didn't know what they were listening to any better (oftentimes worse) than I.  So I just quietly awaited the echocardiogram, and correlated my exam findings with the report of the echo - basically the "gold standard".  Second, I identified a senior cardiologist (Dr. Robinson was his name) who was very well respected in his auscultation abilities - he had trained in the pre-ECHO, phonocardiogram era - and I befriended him.  Many times I would see him in the hallways and say "Hey, Dr. Robinson, I'm seeing this lady with an interesting gallop on her cardiac exam, and I can't figure out what it is, would you mind listening to her with me?" and he would oblige and do a thorough cardiac exam, sitting the patient in all four positions and listening carefully for sometimes upwards of 15 minutes.  When Dr. Robinson rendered his opinion, I could be sure I had the feedback I was looking for.  Once, while seeing a pre-operative consult for atrial fibrillation in the anesthesia suite one day with him and several residents, he identified on the spot, by auscultation alone, a diastolic rumble of MS that had been missed by countless others.  That's the kind of expert that you want to calibrate your decisions to mirror.

In typing this post and trying to remember Dr. Robinson's name, I was reminded by a former colleague of another giant in cardiac examination at Ohio State University:  Dr. Schall.  He was another cardiologist I used to go to for auscultation dilemmas.  It was rumored that he could, by percussion alone, identify the borders of the pulmonary arteries.  Always the skeptic, I dismissed these claims as fancy and lore when I returned to OSU after fellowship.  But I was just told by my colleague, without prompting about the content of this post, the way in which Dr. Schall learned to do this:  he would percuss patients in the cardiac catheterization laboratory and then use flouroscopy as a gold standard to check his percussion against!  Immediate and accurate feedback!  I now believe that it may be legendary rather than lore after all.

So, to maximize your calibration, after you have mastered domain specific knowledge, and begun a program of countless hours of iterative practice, you must seek out feedback and it should optimally be immediate and highly accurate.  If the environment in which you find yourself is bereft of feedback, find ways to increase it.  And make sure the source of your feedback is trusted and accurate - otherwise it's worthless.

And, yes, that's 5 shots in that ragged hole in the photo.