Grading Hospitals
A promising method that researchers hope will boost quality is the “check list” approach, similar to the way airline pilots review a pre-flight list of steps prior to take-off. Note that this approach grades plans on inputs (process) rather than outputs (survival). However, researchers at the University of Michigan studied six types of surgeries with a high risk of death. They found little correlation between death rates and how well hospitals followed process measures.
This is a system-wide problem with quality measures in health care. They are very often measures of inputs, not outputs.
It would be like grading schools based on the years of experience of the teachers, how many advanced degrees they have and how many hours of continuing education they have, instead of grading them on how their students perform.
This could be described and measuring the quality of food based on how many ingredients were used to prepare it rather than how good it tastes. Or at the very least how closely you adhere to the recipe, regardless of how it tastes.
This is a very important post, pointing to what’s wrong with “checklist medicine.”
Isn’t this Econ 101? You grade organizations based on what they produce, not on how hard they work at it.
Checklists and processes can have their place in preventing errors and mortality in healthcare, but I agree that they should not be the actual measure of success. A focus on outcomes measures would give you the information you need to determine whether the checklists and processes you recommended were effective or not. Then, if they are found not to positively affect outcomes, you would need to determine whether the checklists and processes themselves are flawed or whether people simply are not complying with processes and using the checklists. It is not uncommon in healthcare to implement processes only to have them fail due to non-compliance.
In my experience, it is very hard for hospitals to measure outcomes in any meaningful way, due to presence of multiple variables and the length of time needed for most meaningful outcomes to occur. The connection between cause and effect is often not readily apparent. On the other hand, it is very easy to measure a process. Sad to say, these measures have become the ‘benchmarks’ for quality, and are touted as “best practices” for purposes of Medicare/Medicaid reimbursements (watch for this to get worse). In the real world, medicine is actually practiced in that gray zone between certainty and doubt, between art and science. It is frequently not amenable to simple measurement tools.
I actually thinks it makes a lot of sense to look at the inputs. Check this before you rush to judgment:
http://www.leanblog.org/2010/07/standardized-working-toward-zero-infections-and-getting-there/
It’s not about survival rates for very risky surgeries, it’s about making it harder to make the stupid mistakes. In very risky surgeries, the stupid mistakes are just noise. When you are talking about preventing infections, it can provide a measurable improvement in outcomes.
What happens with these bureaucratic idiots is that A checklist becomes THE checklist. As described in the article I linked, shaping the checklist and keeping it up to date is a key step in the process.