Why Pilot Programs are a Waste of Time and Money
President Obama has repeatedly given us his vision of how to lower the cost of health care and raise its quality: Find out what works; then get everyone else to copy it. Toward that end, the administration is making millions of dollars available for pilot programs and demonstration projects. Will any of this work?
Ask yourself this question: Can you think of any other industry where low-cost, high-quality production has been achieved by the government running pilot programs? No? Well, if that approach doesn’t work anywhere else, why would you expect health care to be different?
httpv://www.youtube.com/watch?v=0LcAuQ_AT-s&feature=related
Testing the Waters:
Put your right toe in. Take your right toe out. Put your right toe in. And shake it all about.
As I have previously noted, President Obama’s approach to health care is exactly the same as his approach to education. The only difference is that in education we have been trying to find out what works and then copy it with no success for decades.
Here’s the problem. Both in health care and in education we have lots of examples of low-cost, high quality service. As in other bureaucratic systems around the world, excellence exists, and it’s often known about, acknowledged and even studied. It also tends to have three characteristics: (1) islands of excellence spring up in a sea of mediocrity and they tend to be distributed randomly—they’re not correlated with anything; (2) they almost always exist because of the effort, ingenuity, enthusiasm, energy, and vision of a few people involved in actual production, and almost never are the result of anything that’s happening on the demand side of the market; and (3) (most importantly) they tend not to have any objective characteristics that anyone else can copy.
A study of high-performing hospital regions (by researchers connected to the Brookings Institution) was unable to find any characteristics that could be replicated in a straightforward manner. Some had doctors on staff and paid them a salary. Some paid fee-for-service. Some had electronic medical records. Some did not. (See previous posts here, here, here.)
Another study, reported at the Health Affairs Blog, looked at 12 multispecialty group practices including such high-performing practices as the Cleveland Clinic, the Geisinger Clinic, the Intermountain Medical Group, the Mayo Clinic and the Virginia Mason Clinic. Of those practices, only two employed physicians directly and the other ten paid fee-for-service. Conclusion: There is no relationship between high-quality, low-cost care and the way physicians are paid.
A third study by the Commonwealth Fund examined five high-performing health plans. The only commonalities researchers could find were subjective and qualitative (e.g., “forging and maintaining a strong relationship with physicians,” plus that same idea expressed three or four different ways). Not the sort of things you can put in an operations manual and refer to as “marching orders.”
Instead of a pilot-program approach, at this blog we have consistently argued for an economic approach. Start paying more to the islands of excellence that are higher performers. Start paying less to the low performers. Eventually, stop dealing with the low performers altogether. Encourage every doctor, every hospital administrator and every other provider to come forward and propose different ways of being paid. As long as quality doesn’t suffer, be prepared to pay 50 cents for every dollar the provider saves Medicare.
Finally, search for ways to empower patients — give them control over their own health care dollars and give providers the freedom to repackage and reprice their services in patient-pleasing ways in order to compete for patients and their dollars.
Although I have called this approach the “economic approach,” it is actually little more than common sense.
There is an academic defender of the idea that pilot programs are the key to health reform, however. Writing in The New Yorker, Atul Gawande argued that in the first half of the 20th century, pilot programs run by the Agriculture Extension Service helped spark an agricultural revolution in the United States.
Writing at the Health Affairs Blog, Alain Enthoven showed why Gawande’s analysis was “deeply flawed.” First, the Agriculture Extension Service was working with the winds of the market at its back — helping agriculture go where it was going any way. Second, the pilot program results helped farmers do what they wanted to do anyway. Third, the entire enterprise took decades to achieve. By contrast, the Obama pilot programs hope to show bureaucrats how to force nonmarket solutions on a resistant medical community in the context of a cost trend that will have disastrous consequences if we wait several decades.
Am I right about this? Leave your thoughts in the comments.
John it seems that your characterization “common sense” is rarest of commodities in this administration and its supporters. Mr. Enthoven makes a valid point that can be applied to the rationale political pundits use when connecting political ideology to economic prosperity. Most any political party looks good when in power during an “up” cycle (Clinton in the 90’s). One wonders if it is a lack of common sense or just pure political spin that moves analysts to incorrectly interpret causes and effects.
We can be sure that the Obama regime will take credit for “any” positive signs in the health care issue (or any other for that matter), and cite one of his programs as the catalyst. It appears that the political key is the timing to be “in front of the cameras” so to speak when anything positive is happening. This is much more important to them than actually using “common sense” to lay the groundwork for future prosperity.
It is interesting to compare pilot programs with medical research. Each ultimately aim for similar targets, but pilot programs are more subjective in regards to determining a success. Also they do not address geographical disparities in health care. A program may be successful or cost-effective in one region, but when implemented elsewhere could be a flop. Having a one size fits all solution limits the creativity and flexibility for local health experts to address the barriers to lower cost and quality health care. This is similar to a game of “go fish”…when will you draw the card you’re looking for, and who really knows if you’re playing the best hand?
Sorry Mr. President…go fish
History is full of examples where competition improved quality and decreased prices, and full of examples where price controls created huge inefficiencies.
Health insurance pays for medical services. The 3 inputs are ((cost of medical care X utilization)+Administration). Since every medical procedure and service bought or sold in the US is price fixed by CMS, there is no competition. Therefore healthcare reform continues to try to fix the payment system without fixing the way the product is produced or priced. When the government attempted to make mortgages cheap, the cost of housing went up, until the bubble burst. Reforming health insurance is like putting a corvette body on a Kia chassis.
Thats why we at http://www.medibid.com are actually allowing doctors to package their own product, and price it, and then we let the patient compare quality, pricing, and location and make purchasing decisions based on that.
Pilot programs will be run to prove the proposal (and the proposer) was right. The Pres. thinks this is like testing in alaboratory. It is not. The issues and problems are not knowledge or engineering or planning problems. The issues are third party payment shell games and all of the distortions of price and demand they introduce.
Second, just because a pilot program works in Denver does not mean it will work in St. Louis. So even if some core features are discovered and tested, it would be unwise to universalize them by government fiat. Even worse, just because a set of solutions works today does not mean they will work ten years from now, much less for all of the centuries that government entitlements freeze into place.
John: “Start paying more to the islands of excellence that are higher performers. Start paying less to the low performers.”
I wonder what the criteria would be. What happened when we rated cardiac surgeons by their mortality rates and other simple things? Did cardiac surgeons only take the easy cases?
Great song pairing.
There is a deeper problem – that of the fallacy of composition. Suppose in a society you observer a few fellows making a great deal of money – far more than average – by buying used items and then selling them on Ebay for a profit. It does not follow that GDP will rise if we get everyone to adopt this “best practice.” There are strong complementarities throughout a market economy that make many phenomena surprisingly “unscalable.”
-David C. Rose
John, you assume that measuring outcomes in medicine is easy. There is no standardardized test to identify high quality perfomers. Is it Cost or Outcomes? or Outcomes at what cost? In cardiovascular disease, which has been one of the most targeted areas of health care policy initiatives, measuring outcomes and risk assesing patient populations are very expensive and intellutually difficult processes. It’s hard enough to do at academic medical institutions, much less at the community hospital level – even the large ones. Just adding the outcomes evaluation process to be done in an intellectually rigorous way is not something any insurance company has wanted to pay for. And then there’s the patient performance factor. Do you pay doctors and hospitals less if in spite of their best efforts, heart failure patients won’t stop smoking and lose weight? How many case managers, nutritionists, and hypnotists do you have to hire to get the outcomes the economists/health care experts demand you have in place to get you higher pay? Wouldn’t it be a more rational economic decision to just let somebody else jump through the hoops to get whatever small amounts Medicare’s Independent Advisory Board decides you deserve to be paid? You also assume that there are high quality low cost performers. Some might be lower cost, but there is no such animal. Surely not the Mayo Clinic in Rochester, which is high quality, high cost. And it’s two branches in Florida and Arizona are 30 percent less efficient than the main campus according to my sources. That’s because you can’t transport the good people of Rochester, Minnesota to Florida and Arizona to staff the clinics in those places. It has nothing to do with the doctors and hospital administrators.
Citing government extension agents teaching farmers how to farm back in the 1930s as an example of how pilot projects could work raises some interesting problems.
Back in graduate school, I took a class on economic development. My professor was a self-identified liberal who had worked on economic development projects all over the world. He did not put much faith in government agents whose job it was to improve upon local methods. His anecdotes were mostly from other countries, but he said invariably, the agents did more harm than good.
The main idea he wanted us to learn was the incentives need to be appropriate. Moreover, the ideas need to come from the community, from the bottom up (not top down). The methods used in local communities have often been thought out over generations. His point was that government agents can remove obstacles, but they should not assume they know more than the locals know.
In the case of Medicare pilot projects, it is not in most hospital’s self-interest to lower cost. For a hospital, becoming more efficient means reducing the number of unnecessary (but lucrative) tests and treatments. Firms will need an incentive (such as competition) before they will change their behavior.
I know of one Medicaid pilot that improved quality and reduced costs. It was proposed by groups outside of Medicaid, was tried, and worked. It was extended.
Medicaid bureaucrats are currently promoting step by step regulations that will ultimately do away with the program.
Pilot programs have the potential to learn something by them. IF WE LEARN SOMETHING FROM THEM. In other words, keep what works and abandon what doesn’t. Massachusetts health plan is a good surrogate for Obamacare. Can we learn by its mistakes and make something better? Sadly the answer is no. We see something fail and then just do it bigger as though the problem was that it wasn’t big enough. No it failed. We should do something else.
Even if the pilot program did work, it’s doubtful that the governement could do anything useful with it.
John,
I am writing a short article about cost containment highly consistent with your views including this morning’s commentary that it is hard to enhance the supply of visionary leadership. I have to say though that I would not bet much that even stronger incentives would create that many more islands where they can raise quality and cut cost. I do think better incentives and a safe home would create islands where you could cut cost a lot and only have to cut quality a little. But as I point out in the article it is a good thing to improve incentives whether they “work” or not.
-Mark
Social Security – low administrative cost and high quality service.
Mark Pauly writes: “better incentives and a safe home would create islands where you could cut cost”
I am not sure what that safe home means and would love to read the article, but what I interpret as a safe home is a place where the government and the lawyers are kept at bay from the physician doing a reasonable job.
Why does anyone think that physicians should have trust in a government that has many times broken the law in attempts to break the will of physicians? Why should physicians even trust the incentives that the government will offer? It would be best for government to get out of the health care business.
There is presently promised a significant amount of money for physicians that convert to a specific type of EHR. The physicians in at least one area don’t expect the government to pay its debt and one of the leading experts advising physicians for about 30 years doesn’t believe they will be given what they have been promised. [The private market is providing these certified EHR’s for free]
Good post, John. I posted on it here:
http://econlog.econlib.org/archives/2011/03/central_plannin_2.html
I too like the post. I was trying to figure out if you could ever get a pilot program to work. Devon’s comment about community-up programs was excellent.
Pilot programs don’t work if you are just rehashing a government program. It takes an outside voice to make the necessary changes.
@ Ralph Weber
“Thats why we … are actually allowing doctors to package their own product, and price it, and then we let the patient compare quality, pricing, and location and make purchasing decisions based on that.”
Good for you. That’s the way the system should work.
@ Mark Pauly
“it is a good thing to improve incentives whether they “work” or not.”
Could not have said it better.
@ Charles Johnsen and David Rose
Agree. If you can’t replicate it or if it isn’t scalable, what’s the point.
@ AI and Brant Mittler
I agree that the issue of criteria is a difficult one, and I don’t mean to suggest that it is easily solved. In fact in our proposal we would ask the provider to propose how these judgements should be made. For example, if you propose to Medicare that you want to be paid a different way for a different bundle of services you must (a) tell Mdicare how the two of you will determine that cost has gone down and quality has gone up.
That said, I believe that right now there are some institutions, practices and facilities that have been studied enough that there is common agreement that cost is lower and quality higher than at their competitors.
Linda, what’s the successful pilot program? Cash and Counseling?
@ John, Some call medical a 3rd party payer system, and that is partially true. Have you ever known of ANY third party payer system that works efficiently?
Consider this….
Party A (CMS), fixes prices
Party B (Insurer), pays those prices
Party C (Doctor), accepts payment
Party D (Patient), is the poor sap lost in this murky mess
Our medical system is a 4 party system
@John G. “I agree that the issue of criteria is a difficult one, and I don’t mean to suggest that it is easily solved.”
John, quality is a very subjective term and its measurement depends upon so many things that today it really cannot be accurately measured. The number of variables, changes in circumstances, etc. is astounding. A surgeon might do his best work before he is renown for his quality and his worst work while his reputation is at a high point.
If a cardiac surgeon in the sticks is operating on similar patients as the one at the Mayo and has slightly worse results, does that mean the Mayo facility is better? Selection rears its head as those that are stronger and healthier can survive the trip. The others cannot and are literally guaranteed to have worse results, longer hospital stays and higher costs. Those in a higher socio economic class and can afford to fly are generally better educated and might have higher survival rates based upon their socio economic class rather than the differences in the health care provided.
There might be some risk adjustment, but risk adjustment is in its infancy and cannot guarantee repeated performance.
@ AI
One more time. Suppose you are a doctor (or a hospital) and you want to reprice and repackage your services (say, the way a concierge doctor does).
I would let you explain how your new arrangement will lower the government’s cost and raise the quality of care to the patient. If you can make a convincing case, we should do the deal.
If you can’t — that is, if you say “my deal will lower costs but I have no idea what will happen to quality because quality is too difficult to measure” — then I (being Medicare) would not accept the deal.
John you worte, “Finally, search for ways to empower patients — give them control over their own health care dollars and give providers the freedom to repackage and reprice their services in patient-pleasing ways in order to compete for patients and their dollars.” This should be the first thing, which means you mistakenly got into the “payers” (HMO corporations and gaovernment agencies) paying for “performance” (of quality and outcomes), which others above correctly point out have no reality wheras what is measurable, money spent, is all that counts and is truly measurable. 1st things first, patient power; not vague “performance pay”. Bob
John G. “I would let you explain how your new arrangement will lower the government’s cost and raise the quality of care to the patient. If you can make a convincing case, we should do the deal.
If you can’t — that is, if you say “my deal will lower costs but I have no idea what will happen to quality because quality is too difficult to measure” — then I (being Medicare) would not accept the deal.”
If the deal was being made directly with the patient I would have a bit more faith in your plan’s success, but it is dealing with a third party that is predominantly interested in costs and derives little benefit from the care. That is a relationship that I believe we can both agree does not lead to the best long term decisions and IMO can eventually be quite harmful to the patient.
My assumption is that cost vs quality/access will be the metrics used by the third party to measure success. Quality as we both agree is quite obscure so one can imagine that the third party will be focussing predominantly on costs to the detriment of the patient. The government knows that and the physicians know that. I don’t think the outcome is in doubt.
John–the pilot was Consumer Directed Attendant Support in Colorado Medicaid. It was started before RWJ’s Cash & Counseling but was the same idea. The difference is that CDAS wasn’t designed to be budget neutral. Medicaid recipients shared any savings with the state on a 50/50 basis. This, naturally, has since been eliminated.
Al–if there is sufficient flexibility for people to express the intensity of preferences it is possible that even third party payment can reflect consumer preferences and act as a good agent for patients. Aside from the general success of private US care versus care in countries with government run health systems, the existing example seems to be Howard’s (2008) lone study of kidney transplantation.
His results suggest that in-network transplant hospitals have better outcomes than out-of-network facilities and that privately-insured patients are more likely than Medicare patients to register at hospitals with higher survival rates.
An important question is whether government coverage plans or private insurers act as the better agent for patients, and how the amount of consumer choice allowed affects the answer.
Linda writes: “Al–if there is sufficient flexibility for people to express the intensity of preferences it is possible that even third party payment can reflect consumer preferences and act as a good agent for patients.”
Linda, of course you are right that given appropriate circumstances third parties could act in the best interests of patients, but our experience since cost has become a concern is that the government focus’s on the dollars (it should) and focus’s less on the patient. Thus one can expect that providers, when looking for plans that are acceptable to government, will create plans that benefit two of the three parties and most often the patient won’t be one of them.
Al, I agree that government flexibility at the federal level is an impossibility. And that we wouldn’t want it any other way. Which is why government should subsidize health care by giving people cash or the equivalent.
@ Linda Gorman
“Which is why government should subsidize health care by giving people cash or the equivalent.”
So straight up redistribution of wealth?
Ralph writes, “So straight up redistribution of wealth?”
It is true but unavoidable because if any part of healthcare is to be funded with insurance proceeds (before we even get to Medicare/Medicaid), it is a process of “redistributing wealth”. At any given point the healthy are paying for at least part of the expenses of the sick. If we accept that inevitability then Linda’s suggestion is the best solution.
Put another way, if taxpayers are going to foot the bill do it in a way that places the buying decision in the right hands as often as possible. In this way the decision maker (patient) can use third parties to facilitate the patient’s needs instead of the other way around. This is the only logical solution if one believes that the most efficient decisions are made at the personal level.
@ Frank,
Partially true, but only if you consider a “traditional health plan” to be insurance. However real insurance does a very good job insuring against unsystematic risk where the winners and losers are not known in advance. Used in this way it is a great stabilizer. It’s only when we attempt to make insurance cover systematic risk where it becomes a re-distribution tool.
Linda writes: “Al, I agree that government flexibility at the federal level is an impossibility. And that we wouldn’t want it any other way. Which is why government should subsidize health care by giving people cash or the equivalent.”
Linda, what you say makes sense, but will not happen so a second best solution is in order. That second best solution will have to have at least some of the features that your solution suggests. The patient is going to have to make the provider become more fiscally responsible.
Maybe instead of increasing physician payments from Medicare we should have kept the payments the same and permitted a greater degree of balance billing or permitted balance billing in its entirety. That would put a bit more skin in the game while reducing Medicare’s costs.
@Ralph
Certainly you are correct that “traditional health plans” are not an example of insurance. Insurance should only be applicable for “unknown” risks. But would it not be also true that even catastrophic insurance coverage is in the final analysis a redistribution of wealth from those who have no severe medical services to those who do?
In any event it seems that there will most definitely be a tax supported benefit for some. If that is true is it not better to do this in a way that significantly reduces the influence of all third party payers (especially the government)?
John,
As a wise old man said “there are more people living off of it than benefiting from it. Especially when most focus/task forces are full of people who know very little about what they are providing their “expertise” for.
-Dr Bob Kramer
The market is essentially widespread use of pilot programs working the way Obama would like us to believe a government pilot program would work. Someone tries something, say the iPod, and it works and gets copied. Something else, say the “Wonderfile portable workstation”, turns out to be a flop and goes away. While its true that the flops have a cost, it’s also true that they are a necessary part of the learning process. Who would have thought that the iPod, which costs a lot of money, would actually sell?!
On the other hand, government pilot programs will do exactly the opposite, because they are not subject to the wills of consumers. They face public choice constraints, not market constraints, so we can expect those programs that are most popular with well organized special interests to be the ones to survive. In the mean time, resources are lost debating which programs to fund, where and when to do them, etc. leading to the losses in the “Tullock Rectangle”.
All told, if the government is going to try some hare-brained scheme anyways, it would be less damaging for them to just randomly pick one bad plan, carve it in stone and leave it alone.
You are not only right — you are right on! I know you’ve been on a couple of news cable shows, but I wish you would get more exposure for your ideas and your wonderful historical data. It’s frustrating to me because what you say makes so much sense. I read your blogs trying to put myself in the shoes of a liberal, and you make so much sense I don’t understand how politicians can ignore your insights. Please don’t stop!!
@ Frank T.
Following your train of thought if I buy a gold coin from you it is redistribution of wealth. True insurance is a stabilizer, and risk sharing/shifting method. It’s only systematic risk where winners and losers are known in advance, where adverse selection occurs which creates a problematic redistribution.
I’m sorry that actual effectiveness is not an issue within the Beltway. The left has its win and it’s moved on to other issues.
@Ralph T.
Let me clarify Ralph. As you know “true” health insurance involves pooling losses and developing premium rates adjusted for age and other factors to (as you note) stabilize costs, specifically with regard to catastrophic (or high deductible) health insurance. Because of this process the non claimants are subsidizing the claimants. In the purest sense that is “redistribution of wealth”.
I get and agree with your point regarding first dollar coverage, and how it is counter productive. I only say that if we are to use insurance as part of the healthcare solution (which of course we should) then we should recognize that in its basic concept it is no different in its funding mechanism than Medicare (which is obviously a redistribution of wealth). We “know in advance” that the “winners” are those over 65 and the “losers” are everyone else who pays taxes. The same is true of Medicaid.
If we accept the idea of pooling funds to pay for the health care of certain segments of the population, then why would we not do it in a way that encourages self reliance and cost effectiveness (tax credits and vouchers)? I am simply saying that “redistribution of wealth” is and will always be part of the healthcare system, and we need to control its applications.
Hi there would you mind letting me know which web host you’re working with? I’ve loaded your blog in 3 different
browsers and I must say this blog loads a
lot faster then most. Can you recommend a good web hosting provider at
a fair price? Thanks a lot, I appreciate it!
Also visit my homepage; Gabriele
Everything is very open with a very clear description
of the issues. It was definitely informative. Your site is very helpful.
Thanks for sharing!
Feel free to visit my web blog … Ashli