A new study on another medical checklist is out, this one conducted on noncardiac surgery in eight hospitals in eight cities around the world to ensure a diverse range of clients. As with the now-famous previous checklist – call it Checklist I – this one produced decreases in death and infection rates. What is most interesting about this checklist – call it Checklist II – is that is undercuts one of the supposed insights of Checklist I, namely that shorter was better.
Checklist II is long. Very long. Totaling 19 items. (Recall that Checklist I uses only six.) Does this mean checklist designers need not worry about keeping the number of items to remember to a minimum? Though it would tempting to draw this conclusion, it would be as foolhardy as pronouncing the absolute virtues of brevity based solely on the findings from Checklist I. What is more likely is that checklist length matters less than the overall environment in which each step is taken; the mechanisms in place for enforcing each step are what matter most. This enforcement is all the more necessary since human memory is not reliable enough to count on recalling every step. Indeed, the authors of the Checklist II study point out that, in spite of the medical improvements, “omission of individual steps was still frequent.”
Designers of Checklist II broke the 19 steps into three subcategories, administering the steps at different points in the surgical process. The number of steps and points of administration make the identification of key steps difficult. But the authors explain the overall choice architecture this way:
Use of the checklist involved both changes in systems and changes in the behavior of individual surgical teams. To implement the checklist, all sites had to introduce a formal pause in care during surgery for preoperative team introductions and briefings and postoperative debriefings, team practices that have previously been shown to be associated with improved safety processes and attitudes and with a rate of complications and death reduced by as much as 80%. The philosophy of ensuring the correct identity of the patient and site through preoperative site marking, oral confirmation in the operating room, and other measures proved to be new to most of the study hospitals.
After devising the steps, determining the appropriate points in the surgical process to implement them was the key challenge. For example, the designers encouraged administering antibiotics in operating rooms instead of preoperative wards, “where delays are frequent.”
The paper’s authors are unsettled about the potential of a Hawthorne effect – in which an improvement in performance is the result of doctors’ knowledge of being observed – in the overall results. At the risk of carelessness, for now, this concern is perhaps best left to the academics. Building in an observant whose presence leads to changes in behavior is a perfectly acceptable option for someone who is not so worried about the exact contribution of each casual mechanism. As social scientists know, even trying to isolate the effect of observation can be futile, for in some cases observation is one way to enforce particular social norms, making the combination of the two what influences behavioral change. So while the Hawthorne effect has confounded social scientists for years, everyday choice architects may need not worry so much. After all, the Hawthorne effect itself is just another nudge.