The Checklist Manifesto (2009)
Really interesting book about how checklists are used as a tool in various industries (construction, aviation, finance) and a surgeon attempting to get them introduced in medicine.
It’s a mix of stories and anecdotes about people who are already using checklists, and discussion of the theory and ideas behind them. It has a lot of good and practical information, and delivers it in a concise and easily readable package. It’s also upfront about limitations and risks; it doesn’t present checklists as some sort of easy utopia, but instead discusses the challenges you’ll face if you want to do them properly.
Notes and highlights
Introduction
Two stories from John, a fellow surgeon who had near-misses during surgery:
- He was patching somebody up after a stab wound, but nobody had thought to ask what the weapon was. It was a bayonet that had penetrated far deeper than expected, and the patient started to bleed out on the table.
- A cancer operation that was all wrapped up when the patient’s heart stopped, and it took a while before anybody realised the anaesthetist had given them a fatal dose of potassium and to start the recovery procedure
There are two types of failure:
- We don’t know what we’re doing (inexperience)
- We know what we’re doing, but we get it wrong (ineptitude)
As medical science advances, more and more of (1) is solved. That means more and more of our errors are of type (2). How do we fix them?
The traditional answer is “more training”. The actual answer is a checklist.
Chapter 1: The Problem of Extreme Complexity
Modern medicine is incredible complicated. Procedures involve dozens or hundreds of steps, and you have to get them all right – there’s as much potential to harm as there is to help. And we keep discovering new diseases and ways for the body to go wrong.
P18, how we might have imagined medicine would go:
I think we have been fooled about what we can expect from medicine—fooled, one could say, by penicillin. Alexander Fleming’s 1928 discovery held out a beguiling vision of health care and how it would treat illness or injury in the future: a simple pill or injection would be capable of curing not just one condition but perhaps many. Penicillin, after all, seemed to be effective against an astonishing variety of previously untreatable infectious diseases. So why not a similar cure-all for the different kinds of cancer? And why not something equally simple to melt away skin burns or to reverse cardiovascular disease and strokes?
How has medicine responded to the challenge of managing this complexity? With super-specialisation – you keep slicing into finer and finer categories of expertise, and it takes longer and longer to train as a doctor. Yet errors still occur!
What do you do when super-specialisation fails?
Chapter 2: The Checklist
In 1935, the USAF was doing a competition between two new bomber planes. Everybody thought it was a formality, and Boeing was expected to win the contract – until the plane exploded shortly after takeoff and killed two people, including the pilot. An investigation determined that pilot error was the cause of the explosion – the new plane was so complicated, even a very experienced pilot couldn’t do it all successfully.
Boeing lost the contract, but the USAF still bought a few planes to play with. The pilots didn’t suggest more training – it was hard to get more experienced than the dead captain – instead they reached for checklists. They were eventually able to fly it safely, and it later became the B-17 and a big part of the American WWII campaign.
P34, comparing early flight to the new Boeing:
In the early years of flight, getting an aircraft into the air might have been nerve-racking but it was hardly complex. Using a checklist for takeoff would no more have occurred to a pilot than to a driver backing a car out of the garage. But flying this new plane was too complicated to be left to the memory of any one person, however expert.
P34, on our work being too complex for one individual:
Much of our work today has entered its own B-17 phase. Substantial parts of what software designers, financial managers, fire- fighters, police officers, lawyers, and most certainly clinicians do are now too complex for them to carry out reliably from memory alone. Multiple fields, in other words, have become too much airplane for one person to fly.
P36, on the risks of missing a single step:
Faulty memory and distraction are a particular danger in what engineers call all-or-none processes: whether running to the store to buy ingredients for a cake, prepar- ing an airplane for takeoff, or evaluating a sick person in the hospital, if you miss just one key thing, you might as well not have made the effort at all.
Checklists began to appear elsewhere.
Medicine is another early example. You have to check the four vital signs of a patient, but it’s easy to miss one. It’s not important until one day it is – and nurses started adopting a checklist-style format, actually designing their charts and forms to make sure they never missed a step.
A critical care specialist called Peter Provonost proposed the use of checklists to reduce central line infections – a common complication following surgery. When using the checklist, nurses would be empowered to call out doctors when they did it wrong. It basically eliminated the problem at one hospital, so he tried to get it adopted elsewhere, but medical staff were sceptical. They’re overworked and now they have to do more paperwork? Surely this can only work at rich hospitals.
But he got it adopted at under-resourced hospitals, and he sent in executives to get feedback. This was key, because there were some problems that only they could solve, like the lack of soap on certain wards. It was a big success! The hospital went from below par to above the national average, saving hundreds of millions of dollars in the process. Wow!
Checklists work.
Chapter 3: The End of the Master Builder
Checklists can work, but presumably not in all circumstances. When do they work? When do they not?
There are three types of problem:
- A simple problem, like baking a cake – it’s fairly easy to work out a recipe you can follow
- A complicated problem, like launching a rocket – it’s hard to work out the correct sequence of steps, but once you have, you can repeat that to do it again and again
- A complex problem, like raising a child – prior experience is useful, but the steps that worked for one child can’t be used unmodified to raise another
It feels intuitive that checklists can tackle simple and complicated problems, but what about complex problems.
Gawande noticed a new hospital being built nearby, and decided to find out how the building team ensured it was done correctly. P53:
As I looked up at this whole building that had to stand up straight even in an earthquake, puzzling over how the workers could be sure they were constructing it properly, I realized the question had two components. First, how could they be sure that they had the right knowledge in hand? Second, how could they be sure that they were applying this knowledge correctly?
He went to talk to a civil engineer who’d helped design the building. For a long time, buildings were built according to a “Master Builder”, a single person who oversaw everything. But modern construction is so complicated, it’s no longer possible for one individual to understand everything. The trade has been split into many distinct roles, who work together to make the finished piece.
Medicine is still tied to the idea of a Master Builder; a single physician who calls the shots. It’s been slow to get away from the idea, and the consequence is mistakes and lapses in patient care.
The engineer took him to see a skyscraper under construction. The construction site was managed by joint checklist – the project schedule, coordinated among dozens of experts. They went up the building, to see it being built by layers. Lower levels were more advanced than the top.
P63, a cool fact as they’re wandering round the upper levels:
I did better when we turned our backs to the city and he showed me the bare metal trusses that had been put into the ceiling to support the floor being built above.
Next, he said, will come the fireproofers.
“You have to fireproof metal?” I asked.
Oh yes, he said. In a fire, the metal can plasticize—lose its stiffness and bend like spaghetti. This was why the World Trade Center buildings collapsed, he said. He walked me down a stairway to the floor below us. Here, I could see, the fireproofing material had been sprayed on, a gypsum-based substance that made the ceiling trusses look gray and woolly.
What about expected problems? One thing they noticed was that the floor was sagging, and water was pooling. That wasn’t on the plan! Have do they deal with that?
The project plan include communications – times when the experts have to come together and talk to each other. By ensuring these conversations happen at the right time, the overall plan can be adapted to deal with unexpected setbacks.
This is summed up by a passage on L67:
In the face of the unknown—the always nagging uncertainty about whether, under complex circumstances, things will really be okay—the builders trusted in the power of communication. They didn’t believe in the wisdom of the single individual, of even an experienced engineer. They believed in the wisdom of the group, the wisdom of making sure that multiple pairs of eyes were on a problem and then letting the watchers decide what to do.
Man is fallible, but maybe men are less so.
And then later the point is repeated on P68:
Joe Salvia had earlier told me that the major advance in the science of construction over the last few decades has been the perfection of tracking and communication. But only now did I understand what he meant.
Building checklists aren’t perfect, and stuff can still be missed. But the effect is overwhelmingly positive – buildings in the US are pretty reliable, and faster to build than they used to be.
Chapter 4: The Idea
The construction industry pushes out control and responsibility, and has tried to get away from the idea of a central decision maker. Buildings are too complex for a single person or inspector!
Hurricane Katrina is an example of when centralisation goes wrong. The federal government didn’t have enough information or visibility to understand what was happening, and they couldn’t react to conditions on the ground. This wasn’t a situation that could be managed through central diktat.
Who did a better job? Walmart, P76:
Of all organizations, it was oddly enough Wal-Mart that best recognized the complex nature of the circumstances, according to a case study from Harvard’s Kennedy School of Government. Briefed on what was developing, the giant discount retailer’s chief executive officer, Lee Scott, issued a simple edict. “This company will respond to the level of this disaster,” he was remembered to have said in a meeting with his upper management. “A lot of you are going to have to make decisions above your level. Make the best decision that you can with the information that’s available to you at the time, and, above all, do the right thing.”
They handed out a lot of supplies, helped first responders, and helped people out. Crucially, a lot of this was managed by individual store managers on the ground, rather than central office.
And then on P77:
The assistant manager of a Wal-Mart store engulfed by a thirty-foot storm surge ran a bulldozer through the store, loaded it with any items she could salvage, and gave them all away in the parking lot. When a local hospital told her it was running short of drugs, she went back in and broke into the store’s pharmacy—and was lauded by upper management for it.
This isn’t a story about private vs public sector – many private sector orgs floundered; some public sector bodies like the local fire department did well.
The point is that you can’t run a crisis from central command; there’s too much changing too quickly. You have to give people autonomy in the field, close to where the action is. This is what the construction industry have captured in checklists.
Where else can this apply? How about food? The author went to visit a high-end restaurant, and they were all using recipes – every food station had one. The recipes could be challenged, modified, or changed, but when cooking you stick to the recipe!
Chapter 5: The First Try
The author was invited to be an expert leading a WHO effort to improve global surgery. There were clear issues in surgery around the world, too many complications and mistakes. And the answer isn’t more training or resources – pattern is uniform across countries, both rich and poor.
One early idea was publishing a set of safe surgery guidelines, but this was quickly dismissed, P92:
One had only to take a walk through the dim concrete basement hallways of the otherwise soaring WHO headquarters to start doubting that plan. Down in the basement, while taking a shortcut between buildings, I saw pallet after pallet of two- hundred-page guideline books from other groups that had been summoned to make their expert pronouncements. There were guidelines stacked waist-high on malaria prevention, HIV/AIDS treatment, and influenza management, all shrink-wrapped against the gathering dust. The standards had been carefully written and were, I am sure, wise and well considered. Some undoubtedly raised the bar of possibility for achievable global standards. But in most cases, they had at best trickled out into the world. At the bedsides of patients in Bangkok and Brazzaville, Boston and Bris- bane, little had changed.
There were examples of targeted interventions involving checklists – a study sponsored by a soap company which showed that people will use soap when given clear instructions on how to use it properly, and infections will reduce as a result. (The study was less of a success for the soap company, who were hoping to show their antibacterial ingredient made a different – it didn’t, the behaviour did.)
The John Hopkins hospital adopted a checklist for child surgery, ensuring that antibiotics were administered in the last hour before the first incision – no sooner, no later. It made a big difference, they work!
This “cleared for takeoff” checklist included a fun practical component, P99:
There wasn’t much more to it. But getting teams to stop and use the checklist—to make it their habit—was clearly tricky. A couple of check boxes weren’t going to do much all by themselves. So the surgical director gave some lectures to the nurses, anesthesiologists, and surgeons explaining what this checklist thing was all about. He also did something curious: he designed a little metal tent stenciled with the phrase Cleared for Takeoff and arranged for it to be placed in the surgical instrument kits. The metal tent was six inches long, just long enough to cover a scalpel, and the nurses were asked to set it over the scalpel when laying out the instruments before a case. This served as a reminder to run the checklist before making the incision. Just as important, it also made clear that the surgeon could not start the operation until the nurse gave the okay and removed the tent, a subtle cultural shift. Even a modest checklist had the effect of distributing power.
Teamwork is important in surgery, and the idea of surgeon as star of the operating theatre is dangerous. The power dynamics play out exactly as you’d expect: surgeons think there’s a strong sense of team camaraderie; other members less so.
You can improve teamwork with simple interventions, like asking everyone to introduce themselves first. It turns out to make a big difference in cohesion, and everyone feeling okay to call stuff out when it goes wrong.
Inspired by this, the WHO started designing a surgery checklist. Let’s try it in the author’s theatre! But by the end of the first day in a real hospital, it revealed ambiguities in wording and it was taking long, so they stopped using it. Oh dear.
What went wrong?
Chapter 6: The Checklist Factory
How do pilots use checklists? They’re closer to surgery than construction, so maybe their model is a better fit – they have to make decisions quickly, under intense time pressure. They don’t have time to work through a comprehensive list of everything involved.
The author went to meet somebody who writes checklists for Boeing. He imparted a couple of key ideas:
- Checklists have to be short, precise, efficient
- Only a few key items (5–7 max)
- Laid out clearly
- Use the terminology of the field
- Is it a READ-DO or DO-CONFIRM checklist?
- It has to be tested with real people
Here’s the key passage from P120:
There are good checklists and bad, Boorman explained. Bad checklists are vague and imprecise. They are too long; they are hard to use; they are impractical. They are made by desk jockeys with no awareness of the situations in which they are to be deployed. They treat the people using the tools as dumb and try to spell out every single step. They turn people’s brains off rather than turn them on.
Good checklists, on the other hand, are precise. They are efficient, to the point, and easy to use even in the most difficult situations. They do not try to spell out everything—a checklist cannot fly a plane. Instead, they provide reminders of only the most critical and important steps—the ones that even the highly skilled professionals using them could miss. Good checklists are, above all, practical.
P121:
Pilots […] turn to their checklists for two reasons. First, they are trained to do so. They learn from the beginning of flight school that their memory and judgment are unreliable and that lives depend on their recognizing that fact. Second, the checklists have proved their worth—they work. However much pilots are taught to trust their procedures more than their instincts, that doesn’t mean they will do so blindly. Aviation checklists are by no means perfect. Some have been found confusing or unclear or flawed. Nonetheless, they have earned pilots’ faith. Face-to-face with catastrophe, they are astonishingly willing to turn to their checklists.
The author sat in a plane simulator when a DOOR FWD CARGO warning light popped, similar to United Airlines Flight 811 in 1989. The checklist is short and skips a lot of steps that professional pilots will do anyway, like radioing the tower – it’s equipping to handle an unusual situation, not fly the plane from scratch.
P128:
It is common to misconceive how checklists function in complex lines of work. They are not comprehensive how-to guides, whether for building a skyscraper or getting a plane out of trouble. They are quick and simple tools aimed to buttress the skills of expert professionals. And by remaining swift and usable and resolutely modest, they are saving thousands upon thousands of lives.
Another story from British Airways Flight 38, when ice in the fuel lines caused both engines to fail on the landing approach. The pilots’ instinct was to throttle up the engines and try to restart them – an investigation later discovered that this probably made the problem worse. Instead, the best course of action was to idle the engines and reduce pressure in the fuel lines, allowing the ice to clear. Once Boeing realised the issue, they distributed the new advice to pilots worldwide in 30 days – unheard of in surgery!
Later that year, Delta Air Lines Flight 18 had a similar issue, the pilot and copilot followed the new checklist, and the plane landed without incident. The passengers didn’t even notice. It worked!
Why are doctors slow to adopt new information? P133 offers an answer:
What experts like Dan Boorman have recognized is that the reason for the delay is not usually laziness or unwillingness. The reason is more often that the necessary knowledge has not been translated into a simple, usable, and systematic form. If the only thing people did in aviation was issue dense, pages-long bulletins for every new finding that might affect the safe operation of airplanes—well, it would be like subjecting pilots to the same deluge of almost 700,000 medical journal articles per year that clinicians must contend with. The information would be unmanageable.
And P134 has a fun titbit about airline checklists:
Just as schools or hospitals tend to do things slightly differently, so do airlines, and they are encouraged to modify the checklists to fit into their usual procedures. (This customization is why, when airlines merge, among the fiercest battles is the one between the pilots over whose checklists will be used.)
Chapter 7: The Test
They went back to the drawing board and refined the checklist. They tested it locally, and tried to clear up points of ambiguity, like who should read it out (not the surgeon!).
They stripped it down, with long discussions of what’s in and what’s out. They got it down to nineteen steps, in three stages: before anaesthetic, before first incision, before the patient leaves the theatre.
They deployed it as a trial in eight hospitals, in places with a variety of income levels and resources, including Tanzania, Jordan, Delhi, and London.
Administrators were strongly encouraged not to force it on people, to avoid a negative backlash – instead, introduce it gradually. Let people experience it for themselves, and decide to use it if they want to.
There are stories of people who were won over by it, like on P152:
In London, during a knee replacement by an orthopedic surgeon who was one of our toughest critics, the checklist brought the team to recognize, before incision and the point of no return, that the knee prosthesis on hand was the wrong size for the patient—and that the right size was not available in the hospital. The surgeon became an instant checklist proponent.
P157:
Some skepticism persisted. After all, 20 percent did not find it easy to use, thought it took too long, and felt it had not improved the safety of care.
Then we asked the staff one more question. “If you were having an operation,” we asked, “would you want the checklist to be used?”
A full 93 percent said yes.
The results were clear: a double-digit percentage decrease in post-surgery complications, across the board.
And on P149, a sidebar about gender norms and dating in Jordan:
What I couldn’t work out, though, was how the country’s strict gender divide was negotiated in its operating rooms. I remember sitting outside a restaurant the day I arrived, studying the people passing by. Men and women were virtually always separated. Most women covered their hair. I got to know one of the surgery residents, a young man in his late twenties who was my guide for the visit. We even went out to see a movie together. When I learned he had a girlfriend of two years, a graduate student, I asked him how long it was before he got to see her hair.
“I never have,” he said.
“C’mon. Never?”
“Never.” He’d seen a few strands. He knew she had dark brown hair. But even in the more modern dating relationship of a partly Westernized, highly educated couple, that was it.
Chapter 8: The Hero in the Age of Checklists
The WHO safe surgery checklist is now implemented in a dozen countries and many US states, but it’s often forced on physicians by bureaucrats who are distrusted – so there have been stumbling blocks in its implementation.
Checklists have seen success in other areas – the author spoke to three investors who use a checklist-like approach. When they’re researching a new investment, it’s easy to get emotionally invested before money changes hands, and make stupid mistakes. Checklists reduce the number of mistakes, and help them see what investments are worth making. But other investors aren’t interested in hearing about it!
Here’s one example of how you can get emotionally invested too early from P163:
“You go into greed mode,” he said. Guy Spier called it “cocaine brain.” Neuroscientists have found that the prospect of making money stimulates the same primitive reward circuits in the brain that cocaine does. And that, Pabrai said, is when serious investors like himself try to become systematic. They focus on dispassion- ate analysis, on avoiding both irrational exuberance and panic. They pore over the company’s financial reports, investigate its liabilities and risks, examine its management team’s track record, weigh its competitors, consider the future of the market it is in— trying to gauge both the magnitude of opportunity and the mar- gin of safety.
Then the story of another aviation success: the Miracle on the Hudson. Initially Sully was praised as a hero, because we love the idea of a singular man, the Master Builder. But really this was a success of checklists, and it’s easy to see how it could have otherwise gone wrong.
P175:
Before the pilots started the plane’s engines at the gate, however, they adhered to a strict discipline—the kind most other professions avoid. They ran through their checklists. They made sure they’d introduced themselves to each other and the cabin crew. They did a short briefing, discussing the plan for the flight, potential concerns, and how they’d handle troubles if they ran into them. And by adhering to this discipline—by taking just those few short minutes—they not only made sure the plane was fit to travel but also transformed themselves from individuals into a team, one systematically prepared to handle whatever came their way.
P182:
What rescued them was something more exceptional, difficult, crucial, and, yes, heroic than flight ability. The crew of US Airways Flight 1549 showed an ability to adhere to vital procedures when it mattered most, to remain calm under pressure, to recognize where one needed to improvise and where one needed not to improvise. They understood how to function in a complex and dire situation. They recognized that it required teamwork and preparation and that it required them long before the situation became complex and dire.
This was what was unusual. This is what it means to be a hero in the modern era. These are the rare qualities that we must understand are needed in the larger world.
We expect three things of our professionals: selflessness, skill, trustworthiness. Pilots add discipline to that list. When disaster strikes, they don’t want individual autonomy or a hero mentality – they want the discipline to follow pre-written checklists and work as a team.
Checklist aren’t perfect, and they do need to be regularly reviewed. For example, all aviation checklists are dated, because they’re going to be updated and replaced after publication.
A lot of professions rely on getting incredible individual parts, but not on getting those people to talk to each other. This is bad! We know there’s a way to do this: try a checklist.
P184:
We’re obsessed in medicine with having great components— the best drugs, the best devices, the best specialists—but pay little attention to how to make them fit together well. Berwick notes how wrongheaded this approach is. “Anyone who understands systems will know immediately that optimizing parts is not a good route to system excellence,” he says. He gives the example of a famous thought experiment of trying to build the world’s greatest car by assembling the world’s greatest car parts. We con- nect the engine of a Ferrari, the brakes of a Porsche, the suspen- sion of a BMW, the body of a Volvo. “What we get, of course, is nothing close to a great car; we get a pile of very expensive junk.”
Chapter 9: The Save
The author tried the checklist in his own surgery during the trial period; he didn’t want to be a hypocrite! But he didn’t think it would catch errors – oh how wrong he was.
It caught five errors in three cases in the last week alone.
In the most severe instance, it saved a man’s life. He was doing a routine operation near the man’s heart, sliced wrong and ruptured a key blood vessel. Blood gushed out, but because they’d done the checklist they had the right blood in the blood bank (which wouldn’t have been there otherwise), and the team worked together to fix it. There were some lingering effects from low blood pressure, but he survived.