Ignoring the Obvious? Choosing Suitable Metrics in Evaluating Health Care
One of the biggest problems in health policy is choosing the appropriate metrics to evaluate a complex good like health care. Value is in the eye of the beholder. All too often, the beholder is not the consumer, which leads to an affinity for numeric measures said to be more “rigorous” or “precise.”
As a result, many Medicaid evaluations use population health measures that have as much or more to do with individual behaviors as they do with the action of any part of the healthcare system. Things like number of primary care visits, BMI, cholesterol levels, and blood pressure, are easy to measure provided someone first decides to visit the doctor. And outcomes based on those measures depend upon whether someone decides to diet, exercise, and take the prescribed medications.
Because simple metrics abound, we have a lot of studies evaluating the health “system” that only observe changes in relatively simple and inexpensive treatments that are behaviorally dependent and are provided to a lot of people. Many policy makers are satisfied with this. It accords well with the views of U.S. health care reform advocates who favor more centralized gatekeeping and approve of policies that force people to consume more primary care as a condition of being allowed access to specialists. If people got more primary care, the mantra goes, they wouldn’t need to see specialists.
The choice of behaviorally and genetically dependent medical metrics — prevalence of high cholesterol, high blood pressure, and level of glycated hemoglobin — may even explain why the Oregon Medicaid experiment found that expanding Medicaid had no significant effect on clinical health measures.
In 2008 and 2009, Wisconsin ran another Medicaid experiment. It expanded Medicaid to a group of almost 10,000 very low income, childless, Milwaukee adults. It tracked how they used medical care before and after the expansion.
It found that expanding Medicaid decreased hospitalizations for complications from diabetes, hypertension, congestive heart failure, and dehydration.
While expansion increased outpatient visits by 65 percent, the “majority of the increase [61 percent] in visits was due to increased visits to specialists.” In the discussion, the authors flatly state that “no increase in the use of primary and preventive care was observed.”
Visits to the emergency room rose by 39 percent and almost all of the increase was due to visits for what were called “ambulatory care sensitive” conditions. A problem, as a JAMA article pointed out, is that the same symptoms that can signal a true emergency are often associated with much less urgent problems. It is often impossible for lay people to know which are which. After diagnosis, the emergency room statistics show a non-urgent visit.
Despite the increase in specialist visits, the authors concluded that
Wisconsin’s experience with covering uninsured childless adults, therefore, shows mixed results. On the one hand, hospitalizations declined substantially as did hospitalizations related to ACS [ambulatory care sensitive] conditions. This finding strongly suggests that the underlying health of this population improved as a result of increased access to preventive and primary care. On the other hand, the study finds a dramatic increase in emergency department visits for ACS conditions and no increase in the utilization of primary or preventive care in an outpatient setting. Public insurance coverage seems to be reducing hospitalizations and improving health through increased preventive care, but this care is being obtained in the emergency department rather than in a more appropriate primary care setting.
The study did NOT deal with “childless” adults in the normal sense, but with adults with no dependent children. There’s a big difference.
Adults who have decided against hobbling their lives by breeding call themselves “child-free,” not “childless,” in any case.
Look out! We got a VHEMT sociopath here!
I wonder what the studies would say when comparing “child-free” individuals?
Who would ever think that the choice of metrics affects study conclusions? (Sarcasm)
This article correctly points out how choice of metrics is often done with respect to data that is most easily obtained (or most conducive to the results being sought) rather than that which is most accurate. The Wisconsin Medicaid example is perfect, and the study’s inaccurate conclusion is typical.
People newly covered by Medicaid are rational enough to adjust their behavior according to the increased incentives to visit the ER for non-justifiable reasons. Yet another “unintended consequence” of the Medicaid expansion under the ACA.
And yet a consequence that was publicized that would happen among circles like this blog. Can’t say we didn’t tell them so.
Linda always does a good job of puncturing inaccurate assumptions.
In that spirit, let me try and puncture another one.
Why does it matter how many people use emergency rooms?
If you look at hospital charges, it does seem like a big deal.
But it you look at what hospitals collect, it seems less importnat.
Just for the heck of it, assume we had deep enough national health insurance so that ER use was reduced.
Hospitals would just charge more for full admissions.
As long as a hospital exists, it will send out large enough bills to try and cover its overhead.
It it cannot send out bills for ER use, it will pad the bill somewhere else.
What is the solution?
Well, my solution (which has never been too popular on this blog) is to treat ER’s as part of the fire and police department. Each community would tax itself and there would be no more large bills for ER use, any more than you see large bills from the fire department.
I have always contaended that the taxes should be less costly than forcing everyone to buy more health insurance and subsidizing them.
Bob, you raise a good point. A growing body of contrarians argue the ER is not much more expensive (at the margin) to treat non-emergent patients than elsewhere. By this they mean if we assume the ER needs to be built and staffed 24-hours a day, the marginal cost of 1 person walking in doesn’t boost the costs proportionately. I can easily see both sides to this argument. In theory, ERs could be divided into community health centers and emergent care centers. The community care centers could close earlier and their staff go home. Part of the issue is that ERs are being build with excess capacity to accommodate the influx of non-emergency patients. I’ve talked to small employers who noticed their workers sometimes went to the ER to avoid cost sharing.
I don’t think anyone would agree that’s a good idea.
Under our current system, unnecessary ER use is often used as a gauge to see if Medicaid coverage is changing behavior. In many cases, it doesn’t change behavior or change health status.
What gets measured gets done, as the saying goes. In health care, I’ve heard another saying: It is not achieving quality that increasingly matters, but hitting measures of quality.
“It is not achieving quality that increasingly matters, but hitting measures of quality.”
Oh how true that is!
Good point John! Using the wrong metrics, knowingly or unknowingly, inevitably produces unrepresentative results. Yet, conclusions based on such errant metrics are so commonly believed by pontificators. Even the idea that the US is near the bottom in healthcare performance is commonly accepted!
http://healthblog.ncpathinktank.org/broken-mirror-on-the-wall-on-the-commonwealth-funds-increasingly-frustrating-comparison-of-international-health-systems/
Lies, damn lies, and metrics?
Back to the non-problem with Emergency Rooms:
America is so attached to user fees vs public funding that we make this more complex than it needs to be.
Fire departments definitely do not depend on user fees in 99% of America.
They get one check a year from the city council and that covers the budget.
If fire departments had to charge per fire, and if they were not busy, the accounting cost for a single fire could easily be $10,000.
And we would fret about the high cost of fires.
And people with poor fire insurance or no fire insurance would have to come up with cash, or beg for charitable care, or default on their bill.
Now back to emergency rooms.
Give them one check a year to cover the costs of operation. Quit sending anyone itemized bills.
Yes there will be idiots who get drunk all the time or take too much drugs, and they will get care without paying a user fee.
Well so what. There are people who smoke in bed and their house catches fire and all taxpayers pay to save them.
Call it collateral benefit. Sometimes we can be a little sloppy and it won’t kill us.
Bob Hertz, The Health Care Crusade
I cautiously welcome your idea that ERs be a public service, like fire or police.
However, the problem with ERs is that they treat too many people who do not have emergent conditions. That is discussed frequently at this blog. And, when the government says universal coverage will increase timely care at physicians’ offices and reduce ER use, that has been debunked countless times at this blog.
So, the question is: How to publicly fund only truly emergent cases, and not the other services that are clogging up the ERs?
Also, remember the fire department puts out fires. It does not insure your household effects. For that, you need private insurance.
I grant that a fire is a fire, it is visible to all. Most health conditions are self-contained, the patient is not contagious or flat on their back or with a broken bone sticking out.
The best solution would be to have a subsidized urgent-care clinic very close to the ER or right next to it. These clinics would not be free, but some federal funding could hold down their prices to $25 or $50 in order to see a nurse practitioner or rookie doctor.
I realize that organizationally this gets tricky, and hospitals have been known to dump their overhead costs onto urgent care clinics and make them unaffordable too. But I like the principle.
It gets less tricky if Medicare paid only for lowest cost site of service. But the hospital lobby has knocked that back for years.
A few years ago some hospitals in Dallas had a similar arrangement, where they would triage Emergency Room patients and send some to low-cost primary care clinics. Some hospitals operated their own clinic, while others referred ER abusers to other clinics. I don’t know if they still do this — although it sounded like a good idea. I suspect that the amount of money earned by billing these patients was small compared to the cost of collections. Moreover, the risk of sending someone on who could later claim they were “dumped” could be risky.