This Should be Mandatory Reading in the White House
This is one more installment in a continuing series, brought to you by the universe, entitled “promising pilot projects often don’t scale“. They don’t scale for corporations, and they don’t scale for government agencies. They don’t scale even when you put super smart people with expert credentials in charge of them. They don’t scale even when you make sure to provide ample budget resources. Rolling something out across an existing system is substantially different from even a well run test, and often, it simply doesn’t translate.
Sometimes the “success” of the earlier project was simply a result of random chance, or what researchers call the Hawthorne Effect. The effect is named after a factory outside of Chicago which ran tests to see whether workers were more productive at higher or lower levels of light. When researchers raised the lights, productivity went up. When researchers lowered the lights, productivity also went up. Obviously, it wasn’t the light that boosted productivity, but something else–the change from the ordinary, or the mere act of being studied.
The entire post by Megan McArdle is a must-read: wonderfully written, brilliant analysis.
They can make whatever obesity-fighting changes they want in school lunch menus. It might actually help when it’s reelection time. The more they restrict calories and sweets at school, the higher the revenues at all the convenience stores, which somehow are coincidentally located at the next major intersection up from the entrance to the school grounds.
Yes, well done, Ms. McArdle! This is thoughtful and well explains the reasons we should not be sold on expanding local policy reforms to the national level just because they worked on a small scale.
Saying “promising pilot projects often don’t scale“ is another way of saying you cannot force stakeholders to adopt procedures that are not in their self-interest to do so. Just because Cleveland Clinic, Intermountain Health Care or Mayo Clinic do certain things very well does not mean you can make every hospital perform as well as the top tier hospitals.
In competition, poor-performing institutions are run out of business and the top-performing ones expand and take over their territory. In health care, there is no reward for being a high-quality, low-cost hospital. Neither is there a punishment for being a high-cost, low-quality hospital system. That is why we do not see Cleveland Clinic, Intermountain Health Care or Mayo Clinic expanding the way Walmart, Target, Macy’s and other retailers have expanded.
Interestingly enough, at the bottom of the McArdle piece at The Atlantic was a link to this other Atlantic article with some highly dubious observations, deceptive (vague) statistics, and sweeping conclusions re: cause and effect. Color me skeptical. Speaking of squinting hard and tilting your head to see a connection…
http://www.theatlantic.com/health/archive/2011/12/some-lessons-from-declining-childhood-obesity-in-new-york/250145/
@Devon
“In health care, there is no reward for being a high-quality, low-cost hospital. Neither is there a punishment for being a high-cost, low-quality hospital system.”
I’m pretty sure that the accountable care organization model is one way that is being tried to address this problem.
As for the McArdle article, it is a valid point that successful pilots don’t always lead to successful programs. However, I’m not sure about the logical leap that therefore we shouldn’t try to use the principles of successful pilots on a larger scale. Some of these problems (such as health care) are real pressing issues system-wide, and just leaving them to the free market does not guarantee that they will be solved in any kind of timely manner.
Government always seems to have plenty of “research/studies” that miraculously justifies whatever big government “solution” is working its way done the pipeline. The thing is, once the “solution” is implemented, follow up studies (if even conducted) are either nonexistent, biased or ignored and the solution to the problems of the program is always more money. The only thing that matters is that government grows.