|
|
The Reinertsen Group Blog
Posted on Jan 1st, 2009 at 3:48pm.
During the past 2 or 3 years, a number of hospitals and other health care systems have made signficant, measured improvements in quality and safety. In scores of ICUs, ventilator pneumonias are now rare. MRSA transmissions and other nosocomial infection rates have been cut dramatically. Mortality rates, both raw and risk-adjusted, have decreased by as much as 25% or more. And these improvements in safety and quality have not been an accident. They have been the result of focused attention, fresh ideas, and effective, engaged leadership. Also...they have been achieved during very good financial times, at least for hospitals.
The question for 2009 is: will these improvements sustain through the budget cutbacks that almost all health care systems are now experiencing? Or, as hospitals cut nurse staffing and slash budgets for education, travel, and quality infrastructure, will we see safety levels start to decline, nosocomial infection rates creep back upward, and hospital deaths start to climb? I don't know about you, but I think we're facing a major safety challenge, right now. And I think there are several practical steps every health care system might take to hold the ground that's been gained on quality and safety, despite the current financial crisis. Here are three ideas.
1. Keep the Board's attention on safety: Now, more than ever, it's critical to keep your Quality and Safety report (with measures of your rates of harm, infections, deaths...etc.) first on the Board agenda, not at the end of it. If the rates start to slip, the Board will start asking hard questions, and that's a good thing. Question: where is the Quality Report place on YOUR board's agenda?
2. Talk about it: I've been in far too many senior leadership meetings during the last 2 or 3 months during which staffing and other budget cuts were proposed and approved, without a single voice asking the question: "How can we do this, safely?" I don't want to be the skunk at the CFOs' picnic, but Linda Aiken's work clearly tells us that if all we do is reduce nurse staffing, we will reduce safety levels, and mortality rates will increase. We simply must talk about this issue, and find ways to take waste out of our nurses' and other professionals' work, if we are ever to reduce staffing costs SAFELY. Question: Have you been in a cost-reduction meeting recently? Has anyone raised the question of safety? If not, why not?
3. Go transparent with our measures of safety: A small number of brave organizations such as the Beth Israel Deaconess in Boston publicly display specific measures of "preventable harm" (www.bidmc.harvard.edu). I would bet on those organizations' ability to stay the safety course during a financial crisis. Their commitment is too public, and too important now to too many stakeholders, to be permitted to backslide. On the other hand, if a hospital keeps its safety measures hidden from view, who would notice if the measures started to slip? Question: Has your organization made this sort of public commitment, with highly visible data on measures such as infection and complication rates? If not, why not?
I'd like to hear your answers to these sorts of questions. I'd also like to hear your ideas for how to sustain the hard-won gains you've made in safety over the past few years.
Posted on a snowy New Year's Day in Alta, Wyoming.
Posted on Feb 16th, 2008 at 9:50am.
Has it ever struck you that most of what we work on in quality assumes that the diagnosis is correct--diabetes, colon cancer, mycoplasma pneumonia, rheumatoid arthritis....etc.--and that our principal quality challenge is simply to deliver the right evidence-based care, safely, to the person who has that condition? Have you ever wondered about the quality of the process that resulted in reaching the diagnosis itself?
Here's an illustration of what I mean. If two doctors started with identical patients, at the very same stage in the evolution of the very same disease, and one doctor came to the correct diagnosis over 1 month, in two visits, after $150 of lab tests, and the other doctor took 6 months to reach the same diagnosis, despite 4 referrals to various other doctors, $2500 in lab and imaging studies, and two costly, invasive and dangerous procedures, wouldn't you say that the quality of the diagnostic process was better in the first instance?
In a recent article (JAMA 2008;299:338-340) Eric Holmboe, Rebecca Lipner and Ann Greiner of the American Board of Internal Medicine surface this question of "diagnostic quality" as they consider whether physician knowledge and clinical judgment have an impact on quality. I have argued for years that being Board certified in whatever specialty has relatively little impact on the likelihood that a doctor will reliably execute an evidence-based treatment plan for a common condition. That aspect of quality (delivering the treatment plan reliably for a given diagnosis) is far more dependent on the systems and teams with which that doctor works.
But I have also felt strongly that being Board certified (in the case of the ABIM, passing a fairly stringent test of clinical knowledge and judgment) would make a difference in how quickly, efficiently, and safely a doctor arrived at the right diagnosis, especially for less common conditions, or for unusual presentations of common diagnoses. It is surprising to me that so little work has been done to understand variation in the quality of the diagnostic process, and to assess what factors predict efficient, accurate diagnosis. Here are some of the questions I would like to ask, just to understand the variation, for starters: For patients with the same eventual diagnosis...
- How many visits did it take to reach the correct diagnosis?
- How many referrals to different specialists?
- How many laboratory tests?
- How many imaging studies?
- How many procedures? (biopsies, endoscopic examinations...)
- How long did the whole process take? (this is the "sleepless nights" question)
Once we had some sort of picture of variation, we could begin to study factors that predict the quality of diagnosis, and perhaps even begin to test changes that would improve the quality of the diagnostic process. This arena (the quality of diagnosis) might well be something of a "next frontier" for quality work, and I'm very interested in hearing from anyone who is working on this problem.
Posted on Feb 16th, 2008 at 9:00am.
Patients and families are increasingly being invited into places and conversations that have historically been off limits to them. Many leading-edge organizations now invite patients to sit on improvement teams, include families on rounds, and seat patients on hospital committees. But one hospital has gone where no other hospital has gone before, at least with patient empowerment. St. Joseph’s PeaceHealth in Bellingham Washington now has a patient as a full member of the Medical Executive Committee of the organized medical staff!
According to Marla Sanger, VP, Quality and Process Improvement, the MEC at St. Joseph’s decided to try this out over a year ago, and asked a patient to sit in on the MEC meetings, but to excuse herself whenever the MEC needed to perform a peer review or some other sensitive function. After a few months, the physicians on the MEC started forgetting to ask the patient to leave. And after a year, they made the “Patient Representative” position a permanent feature of the MEC!
The report from Marla is that the presence of the patient has changed the conversation at the MEC. Topics that might have been prominent on the agenda in the past (such as interdepartmental squabbles about privileges, or perhaps being paid for call) just don’t seem to come up as often, and instead, the MEC focuses squarely on its primary function: what needs to be done to improve quality and safety for patients.
Has anyone else done this? What has been the experience? I’ve been asking around and have found no other examples, so I’m curious to know whether you’re aware of others who have placed a patient on the MEC. From my perspective, it’s the most dramatic example yet of “putting the patient in the room.”
Posted on Feb 20th, 2007 at 9:31am.
Most “Quality Dashboards” contain data on rates of hospital-acquired infections, adverse drug events, falls, and other harm events e.g. “central line infections per 1000 line hours” or “falls per 1000 bed days .” Typically, these rates are shown alongside some sort of benchmark rate for that indicator, usually established by analyzing the rates for comparable hospitals, and then displayed as the 50th, 75th, or 90th percentile. It’s not uncommon for the dashboard to display any rate better than the 50th or 75th percentile as “Green.” Expressing data as rates, with benchmarks, allows the quality staff and executive team to answer a question commonly asked by Boards: “How are we doing compared to other hospitals like ours?” Knowing how you’re doing compared to other hospitals isn’t a bad thing.
But some innovative hospitals have started to ask a different set of questions, and to use a different sort of performance indicator to answer those questions. Instead of asking “How are we doing compared to the competition?” they’re asking “How are we doing compared to the theoretical ideal?” (The theoretical ideal is often either 100% or zero).And to track the answer to that question, they’re eliminating the denominator. (For example, they are simply tracking “total number of central line infections each month” and “total # of falls each month.”)
There are five reasons why eliminating the denominators is a good idea. 1. Neither your basic patient population nor your types of service change that dramatically from month to month, (with some notable exceptions for seasonal conditions such as allergies, and for institutions with large seasonal influxes of “snowbirds.”) So a raw count of the number of people who fall in your hospital, or get infected, or have adverse drug events, is a fairly accurate indicator of the burden of harm over time. 2. Any time we make a measurement more complex (e.g by making it a ratio between two measurements) we add measurement error. How accurately are we measuring things like “ventilator days?” 3. If a measurement is not adding value (many denominators fall into this category) they’re simply adding measurement waste. Somebody has to keep track of “line hours.” Is this value-added activity, or not? 4. In order to get benchmarks, deciles and other indicators of comparative performance, we usually sent off our denominator-based measurements to some national or regional data compiler (e.g. Premier, VHA, State Hospital Association…) so that we can get them to send us back our %tile ranking and position. This inevitably introduces delay. How old are the data you show your Board? Six months? Nine months? This isn’t a timely way to oversee and steer improvement. 5. Finally, and most important, many of these denominator-based measurements lull hospital leaders into complacency, in two ways. First, the ratios make the data fairly abstract e.g. “4.9 infections/1000 line hours.” Compare this to what that abstract really means: “14 people doubled their risk of dying in our care last month, because of a line infection that we gave them.” If we want our Board members to understand our data, and to oversee its improvement with urgency, they need to understand it viscerally. Eliminating the denominators helps. The second way in which denominators cause complacency is when leaders look at their dashboards and say, “Hey, we must be pretty good. All our indicators are Green.” To which I say, “And what, exactly, does it mean to be Green?” Being better than the 50th percentile for hospital-acquired infections, in a health care system where 200,000 people incur serious harm every year from these infections, is not “Green.”
So what do I recommend? Try eliminating the denominator, for many of your performance indicators. Track the number of patients who are harmed, or receive the care they should receive , every month, against the theoretical ideal…either 100% or zero. Your data will be more accurate, more timely, and more viscerally meaningful. And that will give you a jumpstart on improvement.
Note: from time to time, you might still have to answer the question “But how are we doing compared to others?” For this you will need denominators. But if you’ve been working with the theoretical ideal in mind, you just might find something interesting when you check your performance against the competition: you’ve blown right past the benchmark!
Posted on Dec 21st, 2006 at 3:38pm.
The IHI 100K Lives Campaign brought an unprecedented level of attention and focus to getting measured results in hospital quality and safety—specifically, 3,100+ hospitals working on 6 measures to avoid 100,000 deaths over 18 months. And the results appear to be stunning—approximately 123,000 people who would have been expected to die in the 18 months between January 1 2005 and June 14 2006, if the risk-adjusted hospital death rates that prevailed in 2004 had simply continued forward, did not die during their hospitalization during the Campaign. The confidence intervals on this estimate appear to be something like +/- 20,000.
Those of us who served as "field workers" in hospitals throughout the country during the Campaign know that this work has only just begun. For many of the measures, in many Campaign hospitals, implementation is nowhere near completion. Most observers expect significant additional impact on risk-adjusted hospital mortality rates, once the six measures are fully deployed. It appears that the 100K Lives Campaign is bringing about a seismic, positive shift in the quality and safety of care in US hospitals.
Or is it? Bob Wachter and Peter Pronovost aren’t so sure. Their paper in the November issue of the Joint Commission Journal on Quality and Safety pointedly suggests that enthusiasm might have trumped science in IHI’s estimates of lives saved, as well as in IHI’s choice of at least one of the 6 measures. Wachter and Pronovost scold IHI for a number of faults: for promoting an intervention that is not known with 100% certainty to be effective (rapid response teams); for ignoring other, perhaps more effective interventions that could have been included in the campaign; for using risk adjustment methods to drive the estimate of deaths avoided; for extrapolating data from only 86% of the hospitals in the Campaign; for using unaudited self-reported mortality rate data; for that taking credit as IHI for quality and safety improvements during these 18 months, when in fact many other things were going on at the same time, including other efforts to promote 5 of the 6 Campaign interventions; and last but not least, for not properly accounting for the fact that hospital death rates had already been dropping for some years. Don Berwick’s response in the same issue of Jt. Comm J. on Quality and Safety is both graceful and helpful, and I recommend that anyone who has expended a lot of effort in the Campaign read both of these papers.
What’s my take on the controversy? I’m not an academic heavyweight like UCSF’s Wachter, or Johns Hopkins’ Pronovost. But I have been out in the field, every week, during the Campaign. And it seems to me that something happened during these 18 months. Here’s my analysis. There were about 800,000 deaths in US hospitals in 2004. Brian Jarman tells me that unadjusted Medicare death rates have been going down about 0.1-0.2% per year between 1996 and 2004 and his risk-adjusted "Hospital Standardized Mortality Rate" for US Medicare deaths during the same period has been dropping faster, at 3- 4% per year, either because of steadily better performance in the face of increasing risk of death in the hospitalized population, or because of more aggressive coding of the risk status of hospitalized patients, or both. Using the most optimistic of Jarman’s rates, against a baseline of 800,000 deaths, a 4% background annual rate of risk-adjusted decline might explain 48,000 fewer deaths during the Campaign. But not 123,000. And the idea that this dramatic change in trajectory is due to "coding creep" or hospital CEOs who are fudging their mortality numbers so that they can collect their bonuses? I don’t think so. Not from what I’ve observed on the ground—at the back door to many hospitals, where there has been a significant, sharp drop in the number of hearses pulling away from the hospital, during the period of the campaign. That has nothing to do with "coding creep."
And as for rapid response teams, it seems to me that the flaw in most of the published analyses is what I would call the "full implementation gap." Most organizations that implement RRTs find that they run into several types of barriers to full implementation. The two principal barriers are 1) nurses don’t want to look like they can’t handle the situation so they don’t call for help and 2) physicians don’t want the RRT called on their patients without having a chance to intervene themselves, first. So, many hospitals "implement" RRTs but really aren’t using the teams fully. Those institutions that are capable of executing these types of changes, system-wide, over a short time, typically notice a sharp, significant decline in code blues and related deaths. Park Nicollet Health Services implemented its RRT at 440-bed Methodist Hospital over 1 week, house-wide, and its data on codes is compelling. (see below) This was in a hospital, mind, that already has a very low Hospital Standardized Mortality Rate.
So it would have been nice to have several randomized controlled trials that were positive for RRTs before recommending widespread implementation, as Wachter and Pronovost would apparently have preferred, but the 100’s of individual case studies like Park Nicollet’s provide a rather convincing evidence set, albeit not RCTs, that convince me that the Campaign did NOT waste a lot of energy and effort of thousands of hospitals when in induced them to implement RRTs.
If something happened to death rates during the Campaign, why did it happen? As IHI’s leaders have said repeatedly, the Campaign was NOT the only factor in any improvement during the last couple of years, but when I look at what’s been going on in the hospitals and states I’ve been working in, the 100K Campaign is way ahead of whatever’s in second place. As for Wachter and Pronovost’s implication that hospitals would have done all these things anyway, without the Campaign (since 5/6 of the interventions were on the CMS or JCAHO measurement sets, or otherwise on some national radar policy body’s radar screen) my only response would be "Yes, but…when would hospitals have done them? In my lifetime? Before I retired?" The 100K Campaign brought about a truly unique sense of urgency to the national improvement agenda.
So I think something happened, and that the Campaign had a lot to do with it. Clearly, I’m not the one to settle either the "whether" or the "why" argument, and so I will leave the debate to the health services research experts, for whom this issue will no doubt generate lots of grant requests for years to come.
But while the experts worry about their grants, and their publications, I worry that many doctors, particularly academics, will seize upon the questions raised by Wachter and Pronovost and use them not as reasons to learn, but as reasons to avoid taking action, on ANY of the IHI Campaign Planks. In other words, just as the media might have presented an overly enthusiastic representation of the Campaign results, I worry that physicians’ natural skepticism will produce an overly pessimistic reading of Wachter and Pronovost’s paper, until every last question is answered by the academics. Again, my impatience comes through. "When will we get these perfect answers? What is the harm in NOT acting?"
Finally, I must say I was puzzled by the tone of Wachter and Pronovost’s paper. By describing Don Berwick as "chanting" the mantra of the campaign ("Some is not a Number, Soon is Not a Time"), by implying that IHI somehow had a "conflict of interest" in the Campaign (I’m still scratching my head on that one), and in a variety of other little ways throughout the paper, the authors convey a tone of distainful academic detachment at best, and a sort of eye-brow-raised disapproval at worst. A lot of people must be asking, "What was that all about?" IHI didn’t ask the academics’ opinion? IHI generated too much enthusiasm for improvement, and got too much of the limelight?
Perhaps we should all pause and paraphrase Harry Truman: "It’s amazing how many lives you can save when you don’t care who gets the credit." Our patients need both our science, and our enthusiastic application of the science.
|
