The Dartmouth Atlas Controversy
The Dartmouth Atlas shows wide differences in Medicare spending across health care regions that cannot be explained by differences in health care outcomes. It has been touted by the Obama administration as Exhibit A in the case for health reform. However, The New York Times the other day took the Dartmouth researchers to task in a front page article and columnist David Leonhardt piled on. I also weighed in at this blog and the Dartmouth folks defended themselves here.
I think the work of Elliott Fisher and Jonathan Skinner is very important. But behind the very public to and fro about their work is a very important policy issue that all sides allude to and none are confronting directly. To wit: Can we (meaning the government) use statistics on Medicare spending to force the health care system in general, and Medicare providers in particular, to deliver more efficient, higher-quality care?
The weight of the evidence I believe is fairly clear: No, we cannot.
Before getting to details it may be helpful to remind everyone that this health care discussion is very similar — almost word-for-word similar — to the public policy discussion over public education that has been underway for a quarter of a century. See my previous post here.
Education researchers have conducted Dartmouth Atlas-type investigations and documented wide disparities in outcomes among public school students that cannot be explained by student socioeconomic characteristics or by any inputs controlled by the governing authorities. Good education outcomes are not correlated with the amount of money we spend, pupil/teacher ratios, etc. If anything, the districts that spend the most money seem to have the worst outcomes!
Why is that? In general, a bureaucratic system is one in which normal market forces have been systematically suppressed. In such an environment, there tends to be a sea of (relative) mediocrity, sometimes punctuated by little islands of excellence. Further, the islands of excellence tend to be randomly distributed. They do not correlate with much of anything.
This picture not only describes most public school systems around the world, it also describes most health care systems. (In fact, I can’t think of a single exception.)
Now for 25 years, education policy experts have been asking this simple question: Is there a way for the funders of care (the buyer side of the market) to get low-performing schools to produce as well as the high-performing schools? In other words, can we use the power of the purse, the power of authority or any other governmental power to replicate excellence?
The answer appears to be: No. Every example we have of a good school is an example of excellence achieved because of the enthusiasm, energy, leadership, vision, etc. of people on the supply side of the market. We have no examples of good schools created by federal judges, school boards, city governments, state governments, etc. (unless you count magnet schools that siphon off all the best students).
The same thing is true in health. Recently, a group of researchers from Brookings and other think tanks identified some of the best-performing health regions in the country. How many of these were created by Blue Cross? Zip. How many were created by employers? Zero. How about Medicare? Nada. Further, the researchers could find no common parameters that could be easily duplicated. That is, some plans were capitated; others were fee-for-service. Some had electronic medical records; others did not.
Now back to Dartmouth. Based on Medicare data alone, Dartmouth researchers not only found wide discrepancies in spending, they concluded that up to 30% of spending could be saved if the rest of the country practiced medicine the way the lowest-spending areas practiced. Based on this observation, OMB Director Peter Orszag claimed that $700 billion could be saved if Medicare changed the way it purchases health care. (Notice the slip from a purely hypothetical statement that involves no practical implementation to a completely unsupported claim about what policy can actually achieve.)
Now aside from the obvious observation (at least it’s obvious to me) that we don’t know how to replicate low-cost, high-quality institutions, critics have made several points. The New York Times showed that low-cost hospitals can have poor quality as well as high quality. Some have argued that higher spending institutions often have better outcomes. Others have objected that focusing only on Medicare data ignores non-Medicare spending at the same institutions.
In their research on Medicare spending, Thomas Saving and Andrew Rettenmaier found that:
- Medicare and non-Medicare spending tend to be substitutes for each other. Where Medicare spending is high, private spending tends to be low and vice versa. This implies that much of what the Dartmouth Atlas calls excessive spending may just be cost-shifting to Medicare in areas where private payment is low.
- Among states, we could hypothetically reduce spending by 25% if all states practiced medicine as efficiently as the five lowest-cost states. But after adjusting for socioeconomic and other differences the savings drops to only 10%. When all health care spending is considered, the potential saving is only 5%.
And that 5% assumes you do everything right.
So how can we achieve efficiency in health care? Recognizing that all our examples of low-cost, high-quality care originate on the supply side of the market, it is to that side we must turn. Rather than have third-party payers create more rules, regulations and perverse incentives, we must liberate the providers from the rules, regulations and perverse incentives that already exist. Rather than impose such demand-side initiatives as managed care, pay-for-performance, value-based purchasing and the artificial creation of HMOs, PPOs and ACOs, etc., we must let the providers show the payers how to get the job done.
The providers have far more information than paying institutions will ever have. They have professional pride, training and experience. And (not to be overlooked) they actually know the patients they treat.
I have written about how to do this elsewhere.
This is exactly the right distinction to make. Dartmouth claims about the 30% savings are completely hypothetical. Orszag’s claims are claims about practical public policy. And you are absolutely correct. Orszag’s claims are totally unsubstantiated.
I agree with Ken.
I’ve said all along that telling a high-cost, low-quality hospitals to merely practice medicine like Geisinger, Mayo or Intermountain is akin to telling a floundering Kmart… “just do it like Walmart”.
It’s not that easy. Kmart probably would switch strategies and operate like Walmart if they have the knowledge and resources. But they don’t. The same is true of inefficient hospitals – many of which are protected by powerful political interests.
Furthermore, many of the changes we believe Medicare providers in high-cost regions should adopt would substantially reduce their income. Hospitals not competing on price have little reason to adopt business processes that reduce revenue. To make matters worse, individual doctors that order the redundant or unnecessary tests profit from their use and have little incentive to change even if the hospital asked them to.
I think the results from the Saving/Rettenmaier study are devastating. Somebody needs to send the study to the White House.
John,
You wrote — “Based on Medicare data alone, Dartmouth researchers not only found wide discrepancies in spending, they concluded that up to 30% of spending could be saved if the rest of the country practiced medicine the way the lowest-spending areas practiced. Based on this observation, OMB Director Peter Orszag claimed that $700 billion could be saved if Medicare changed the way it purchases health care.”
Orszag is a fool. All of the health systems he admires are subject to the EXACT SAME payment system as those he despises. Clearly the payment system is not the cause of either excellence or mediocrity.
I agree with Greg.
John – great post. One thing I’d modify, though, is the very end. Providers actually usually don’t have good information on patients. The whole electronic medical record industry partially addresses this, but those tend to be physician and/or hospital system centric. That means they don’t capture the whole picture either, and when the consumer moves all is lost. The only logical point around which to aggregate data is the consumer. And properly motivated, consumers have shown they might actually use it.
Let’s not overlook the work of Richard A. Cooper at the University of Pennsylvania who showed that more spending equals better care and that Medicare is not a good proxy for total health spending in an article in Health Affairs. He provides a nice summary of some other articles critical of the Dartmouth approach at http://buzcooper.com/2010/02/18/the-death-of-dartmouth-but-whos-to-blame/
Joseph J. Doyle had an interesting paper that compared the experience of visitors who became ill in Florida. He found that people who became ill in high spending areas had significantly lower mortality rates than those who became ill in low spending areas.
So much for the Dartmouth and ObamaCare claim that higher health spending is simply wasted.
There are few additional facts that need to be considered. First and foremost, Medicare data does not allow for the use of physician identifiers. The public (employers, coalitions, etc.) is not allowed to see nor use the physician identifiers in any analyis. (See the law suit by Consumers’ Checkbook.)
Until there is individual accountability, the system will change little. It has been shown that data can change everything. For example the work done in the 1990s around coronary bypass surgeries in NYS hospitals. When the data became public hospitals changed their practices. Or the publication of hospital infection rates has changed the landscape in infection control in hospitals.
In order to get orders of magnitude of improvement in the practice of medicine, the elimination of waste and lower costs we need to have tort reform and transparency to data about individual practitioners. (www.hcbn1.com)
John,
Good post. One correction, though. If, as you say, “the districts that spend the most money seem to have the worst outcomes!”, then you can’t also say, as you did in the previous sentence, “Good education outcomes are not correlated with the amount of money we spend.”
Best,
David
Excellent points. In analyzing medical costs, we also have to account for real estate costs, physician office expense in urban area. Rental and leasing costs appear to be rising in my neighborhood. And how does a hospital or doctor’s office attract nursing personnel. Probably nurses want more income also.
I have surveyed hospitals with the old JCAH as part of the California Medical Association Medical Staff survey. I observed that hospitals and medical staffs have differing and unique personalities and characteristics that are very hard to quantify. Some surgical programs are weak, some are strong.
Great post John. The analogy to public education is compelling.
John,
Good post as usual. The free market concept as you have referenced in a number of posts is beautifully articulated in Michael Porter’s and Elizabeth Olmsted Teisburg’s atricle in the Harvard Business Review way back in June 2004– “Redefining Competition in Health Care” and their subsequent book on this subject. Too bad the administration did not read it and incorporate the concepts in their bill. All the best.
I apologize, the author’s last name is spelled Teisberg.
One significant error in this post. There has been SIGNIFICANT demand-side impact on the health care supply chain in at least four of these markets: Rochester, Portland, Asheville and Everette…and likely some of the others. The best results in health care happen when the buyer is clear about the value they want to buy, and works collaboratively with suppliers who organize to reduce waste and align processes to value creation. Sometimes, health care leadership is strong enough to drive superior value without reciprocation from purchasers, but those providers will be the first to tell you that it is far more sustainable to have purchasers reward quality and value with their benefit policies. And, in other markets, employers are pushing for value and–finding it lacking in their own market–they seek it elsewhere. A very recent example is the reasonably high-profile move of an employer group in Las Vegas, telling local hospitals it would transport employees to Utah to get care if they could not figure out how to improve quality.
Do not count employers out. When they manage their supply chain, they are very, very powerful.
Response to David Henderson: Good catch, but you’re not quite right. The four or five largest school districts in Texas probably spend the most per student and probably have among the worst results. But this doesn’t mean that there will be a negative correlation between spending and results for the 1000 + districts statewide.
Response to Chuck R: I do not regard common sense buying as telling doctors how to practice medicine.
Steering employees away from high-cost, low-quality institutions is common sense. But that’s not the same thing as figuring out how to replicate excellence.
Linda Gorman writes,
“So much for the Dartmouth and ObamaCare claim that higher health spending is simply wasted.”
I don’t know that they (Obama and the ilk) are not telling us what they really believe in this situation. In their world anything spent over the specified budget is “a waste”. I don’t think decreased mortality that you note in the higher spending facilities is a consideration for the Leftists.
I couldn’t resist sending this hyperlink to you. Another example of responsible leadership and management in the health insurance industry – I think not – blue cross further demonstrates their arrogance! See that’s why health insurance continues to rise and rise (I’ll admit to this being one of the reasons) for the consumer – to further the greed of insurance company leadership. I believe this salary beats Aetna, Humana, United Health and most of the others.
John,
Touche.
Best,
David
The simple truth is that none of these references to cost and value studies and measures has much meaning at all in the absence of a valid external, objective metric for measuring what the heck any medical procedure “costs,” let alone figuring what it is “worth.” Bill masters and Medicare schedules are worthless tools for establishing how any delivery system, methodology, practice model, or other reimbursement scheme actually produces in savings or relative efficiency. The only thing that can do that is “fixing the background metric” to borrow a physics analogy. A more-or-less free market system wherein balance billing is a “normal” feature and there is price rationality, transparency, and consistency is the only mechanism that I believe can provide this in the absence of totalitarianism in medical care (universal provider), which is what “Medicare for all” would be, and where ObamaCare leads us to.
John, great post here… Given a patient’s hospital stay is defined by a few very important events separated by vast expanses of nothing valuable (TV watching, wandering around, etc…), I think there is an argument that iterative clinical efficiencies could be easily eclipsed by simple operational improvements. Hospitals, after all, can be likened to extremely inefficient factories (raw goods in are ill patients… finished goods out are healthy ones)… But they are silo’d and the awareness of one care provider’s actions and status is often a mystery to the others in the hospital. I’ll bet an operational transformation could not only greatly eclipse the fiscal benefits of iterative clinical improvements but could actually sustain or advance clinical quality. Without a quantum change in the operating efficiencies of hospitals generally, quality will inevitably decline.
I like the comparison to government schools as well, but I can go one better. Where does the government have an even greater monopoly, along with the added benefit of unquestioned constitutional authority to exercise it? Plus a culture of shared values unequalled in our society and a degree of central command and control reinforced by punishment unacceptable to most of us?
The answer, of course, is the military. But nobody would assert that every infantry regiment operates with the same “quality” as every other infantry regiment in the Army, nor that every destroyer operates with the same “quality” as every other destroyer in the Navy, etc. These units are bound by rules and regulations, but the result is not equal outcomes. It comes down to leadership, which can never be equally distributed.
Have you read James Q. Wilson’s “Bureaucracy: what government agencies do and why they do it”?
[…] John Goodman compares the debate to that over public education, and says: I think the work of Elliott Fisher and Jonathan Skinner is very important. But behind the very public to and fro about their work is a very important policy issue that all sides allude to and none are confronting directly. To wit: Can we (meaning the government) use statistics on Medicare spending to force the health care system in general, and Medicare providers in particular, to deliver more efficient, higher-quality care? […]
I do not even know how I ended up here, but I thought this post was good.
I do not know who you are but certainly you’re going to a famous blogger if you aren’t already 😉 Cheers!
Here is my website: internet marketing