EBRI’s Puzzling Study
Our friends at the Employee Benefits Research Institute (EBRI) have come out with a new study on consumer-directed health that raises far more questions than it answers. The study was published in Health Affairs, but EBRI also offers an extensive appendix with additional information.
The abstract sums up the lessons thusly –
We explored effects of consumer-directed health plans on health care and preventive care use, using data from two large employers — one that adopted a CDHP in 2007 and another with no CDHP. Our study had mixed results relative to expectations. After four years under the CDHP, there were 0.26 fewer physician office visits per enrollee per year and 0.85 fewer prescriptions filled, but there were 0.018 more emergency department visits. Also, the likelihood of receiving recommended cancer screenings was lower under the CDHP after one year and, even after recovering somewhat, still lower than baseline at the study’s conclusion.
The authors seem to be saying that CDHPs aren’t working all that well and are discouraging the use of preventive services. They also suggest that, by discouraging physician office visits, CDHPs may increase the use of emergency departments. They recommend exempting physician office visits from the deductible.
Maybe, but this research is an awfully thin reed to hang such a recommendation on.
It completely fails to consider what, if any, employee education this employer offered. Did the company just suddenly change benefits or did it work with employees so they would understand the new program? Did it offer patient information support once the program was in effect, or did it leave people in the dark? The authors have nothing to say about any of this.
The authors note that one of the motivating factors in changing to CD Health is financial — employers are trying to save money. But there is not a word in here about the financial effects on the company, or on the workers.
The authors seem to believe that preventive services are unquestionably a good thing, but they acknowledge in a footnote that during the time of this study the recommendations for breast cancer and cervical screening were changed substantially, so even the experts were confused about what to do.
Finally, the authors look solely at the numbers of visits without any consideration of the value of the visits. For instance, they measure the number of prescription drug fills without any consideration of generic substitution, or of replacing some physician office visits with more efficient retail clinic visits for routine services.
The Health Affairs article presents the information on the use of services in a table of raw data –
This is curious because Health Affairs is usually far more graphical in presenting data. And it turns out that such a graphical presentation is readily available, but not unless you go to page 13 of the online appendix
Now that we can see the data in chart form, we have a better idea of what is going on here.
There was a substantial first-year dip in discretionary services among the people who were switched to a CDHP. This should not be at all surprising. It always takes a while for people in a new plan to figure out how to use it — what’s covered, how to file claims, what providers are in the network, and so on. Plus, when a change is imposed on employees by the company, there is a great deal of first-year skepticism and even hostility. After that first year, things seemed to settle down quite a bit.
The study authors seemed concerned about the use of preventive screening, but these charts show that the CDHP enrollees had a higher use of screening services than the PPO enrollees in the last three years of the study period. Perhaps the authors should be more concerned about why PPO enrollees had such a dramatic reduction in their use of screening services over the years of the study.
The authors might also wonder about the spike in inpatient hospital days among the PPO enrollees in the first year of the study. Was there an epidemic that hit only PPO people that year? And why are PPO enrollees’ use of emergency departments steadily dropping? Was there a change in available emergency services for this population?
It is also curious that, after the first-year dip, the rate of Rx fills for the two populations is rising almost identically. This makes our earlier question about generic substitution all the more important.
Like I say, this research raises many more questions than it answers. It’s a pity the authors chose to address only a few of them.
It seems like the Employee Benefits Research Institute has quite a few holes in their study to provide a recommendation.
“Did the company just suddenly change benefits or did it work with employees so they would understand the new program? Did it offer patient information support once the program was in effect, or did it leave people in the dark?”
How does employee knowledge change the efficacy of the program?
Nigel, the effect can be enormous. When you are giving people more power to make their own decisions, they need to have information about what to do with that power. First they should know about the program generally — that they will now be expected to pay for the service directly, that many provider offices may be confused at first. And then they need to know about finding the best treatment options, how to shop for price, how to request generic substitution, and so on. Good patient education is vital.
“Finally, the authors look solely at the numbers of visits without any consideration of the value of the visits. For instance, they measure the number of prescription drug fills without any consideration of generic substitution, or of replacing some physician office visits with more efficient retail clinic visits for routine services.”
how many people use retail clinic visits for routine services? You would have to prove whether or not that is a significant number to prove that the study is not effectively analyzing the problem.
No, I don’t have to prove anything. I didn’t do the study. You are right that we don’t know how many people use retail clinics (at least in this group). But if the emphasis is on the number of visits, it would be valuable to know whether people are finding more efficient ways to treat ailments. For example, perhaps some providers demand follow-up visits in all cases while others want follow-up only if the problem recurs.
“There was a substantial first-year dip in discretionary services among the people who were switched to a CDHP. This should not be at all surprising. It always takes a while for people in a new plan to figure out how to use it — what’s covered, how to file claims, what providers are in the network, and so on.”
It takes an entire year to figure out how to use your health service, I think it is a much more likely explanation that it is financial costs that are decreasing usage of healthcare services.
Yes, Calvin, for a lot of people it does take that long, especially if there was not much employee education at the start. People don’t tend to study their new contracts, preferring to figure it out once they have a need.
“It is also curious that, after the first-year dip, the rate of Rx fills for the two populations is rising almost identically. This makes our earlier question about generic substitution all the more important.”
Excellent point! That does make the substitutions a bit suspect.
If we look at the raw numbers (exhibit 3) we see that the CDHP sample had significantly higher physician visits than the PPO population (3.0509 versus 2.8268). Yet the number of preventive services, i.e. cancer screenings, was exactly the same.
That suggests that the workers in the CDHP group (a mid-western manufacturing business) were significantly more likely to see the doctor when they did not need to, than the median U.S. worker. The employer likely recognized this and that is what prompted the full-replacement approach.
The other (national) employer, never introduced a CDHP, which suggests that its employees were less likely to see physicians, for whatever reason.
We do not really know the characteristics of the two samples, but it looks like there were factors beyond the comorbitity and demographic factors that were paired.
Sadly, opponents of CDHP set an impossible standard. They will always reject CDHP as ineffective unless it’s proven that CDHP enrollees are: 1) sicker than average upon enrollment; but healthier thereafter, 2) poorer than average; 3) increase the use of preventive services, outpatient visits, and drug utilization; while 4) drastically lowering inpatient admissions, length of stay and total costs.)
Of course, that’s an impossible goal to achieve.
Yet, we know that a fair amount of physician visits are of little value. Some inpatient days have little marginal value. Many drugs are of little value. The vast majority of preventive services have no value (unless you’re one of the few people unlucky enough to discover you have a medical condition).
CDHP plans need enrollee education. They need the knowledge that prices vary and the tools to compare prices. I’ve known people with high-deductible plans who were perfectly willing to compare prices if only they knew it was possible and had a mechanism to do so.
Didn’t read the study as I recall the ideological agenda, but… if the number of more “discretionary” doctor visits is down by 25%, the denominator of the fraction for measuring ER visits (presumably less discretionary; more urgent problems)is also smaller. Is that a factor in their calculation? Is there any change in the acuity of services… in either venue, the Rand study would predict not. As a place to receive care, the ER isn’t where most cost conscious buyers will/ should go?
I only glanced at the blog, but I didn’t notice any comment about patients stocking up on services prior to a change in insurance. The dips might represent patients getting these preventative care services under their old insurance so they could save the money on one episode of care. Same for pharmaceuticals.
Al, that might be plausible, except that the first-year utilization is for the year prior to the change, and it is nearly identical on every measure except physician office visits.
Greg, I must be missing something. My assumption, taking breast cancer as an example, is that 2006 was the base year and in year 2007 breast cancer screening fell from 64% to 56%. Some of that drop might be due to patients getting their 2006 screening later in the year and restarting it in 2008 so the time span was just slightly longer. Same with the others and pharmaceuticals where sometimes people will keep refilling them as early as possible so they can delay buying them for one year reducing the payment for the deductible by one year.
What comes to mind is that something is either wrong with the research design or the program implementation. A fair number of studies — starting with the RAND Health Insurance experiment — have all found medical spending can fall 30% when most populations are exposed to significant cost-sharing. If the treatment group in this study didn’t out-perform the control group, something is amiss. Either the control group wasn’t really a matched control. Or the firm implementing the CDHP didn’t really implement it correctly. Economic studies — and common sense — all confirm patients act more like consumers when they have an incentive to do so.
Devon, right. And I would think a researcher would look into the causes of such an anomaly. Perhaps in this case the researchers went into the project expecting these results, so didn’t catch the problem.
My girlfriend has a high deductible health plan. Her spending was high last year due to a health problem she experienced. That said, she did exercise discretion to lower spending compared to what someone who didn’t have a CDHP would have done. She looked for generic drugs, she called around about diagnostic procedures. She went to free standing clinics rather than hospital-based clinics.
When deciding whether to switch to an CDHP a few years ago (basically a higher-deductible version of the Blue Cross plan I already had), I noticed that the seemed a better choice if my expenses were very high or very low, with a narrow band where the traditional plan was slightly better.
Anecdotally, switching to a CDHP motivated me to ask my physician about dropping a Zetia prescription, which had by that time become controversial. I guess that was my “one fewer prescription.”
Bart, exactly right. And this has been one of my long-standing arguments. Mid-level spenders will be financially disadvantaged, but that is exactly the population we want to target. They spend enough that it makes a difference and they are well enough that they can influence how much they spend. These people with chronic conditions are at the center of the cost problem.
When critics complained “only the healthy would benefit from MSAs/HSAs,” we used to explain that yes, the healthy would be better off but so would the sick. Only those in the middle would be indifferent financially – but they may enjoy the added flexibility.
Back around 2001 when I was enrolled in a pre-Archer Golden Rule Medical Savings Account, my doctor gave me a 30-day supply of Claritin as a sample. I soon discovered a prescription would exhaust virtually my entire MSA allocation (~$1,200) for the year. Back then, if I didn’t use my MSA up by the end of the year, my employer would cut a check for the balance net of tax. I quickly found an over-the-counter combination of Actifed and Sudafed that worked as well for 5% of the Claritin price.
There is no question in my mind that CDHC works. The problem is that patients have been insulated so long from participating in their medical decisions that the industry isn’t competitive and comparison shopping is more difficult than it would be otherwise.
All these results appear to be the same, considering a reasonable margin of error… especially during the last two years after the CDHP people have settled in.
This paper accordingly becomes an argument FOR CDHPs because they must have lowered employer costs or they would not have studied them.
Hmmmmm. Just wondered what happened to the confidence intervals for the proportions given in the table….
Too bad we’re still fighting about whether HSAs “work.” This study shows that, like most financial incentives, higher deductibles have a large initial effect followed by some regression towards but not all the way to the base case. They moderate overall medical spend and give many (not all) participants the opportunity to keep a lot more of their money. Any observed decline in preventive services is not due to plan design since preventive is free in essentially all HSAs.
Practical people have moved on from the politics and are rolling out HSAs at a faster pace than ever before.
Highly credible researchers like EBRI’s (disclosure: I am an EBRI trustee) can help us make HSAs work BETTER. We all have our biases, but I welcome EBRI to this field.