Can Anything Be Trusted in This Administration?
The new announcement that the Census Bureau is completely changing its Current Population Survey (CPS) questions about health insurance coverage (see previous post here) is devastating for those of us who do health research.
We have all known for years — decades — that the CPS count of the insured isn’t especially accurate. The questions it asks are about full-year coverage but people tend to answer based on their current status. It chronically under-reports Medicaid enrollment — the actual head count from Medicaid programs is always higher than indicated in the survey. The same is probably true for employment-based coverage. It has often been criticized for being weak on foreign language questions. Massachusetts, for example, has a significant population of people who speak Portuguese and that state thought the CPS failed to capture those people.
The Census Bureau recently (in 2007) revised its numbers because the software was misallocating people who reported that everyone in their family was covered. More on this below.
But despite all this, the CPS numbers were very useful. It is an enormous survey of 78,000 households, and since it also asks about employment and income, it is possible to look at very detailed demographic categories. The error rate appears to occur throughout the data, it is not concentrated in any one demographic group, so it is not a factor in comparing sub-groups.
And most importantly, it has been going on since 1987, so it is possible to measure changes over time — during, before, and after recessions, before and after new initiatives like the SCHIP program.
And the state-by-state numbers are invaluable. Want to know how Massachusetts fared under RomneyCare? The CPS is the only place to go.
To drop everything that has gone before in favor of a brand new set of questions is unprecedented. And to do so at the very moment of the biggest revision of health care in American history is completely irresponsible.
We simply will not be able to compare before and after ObamaCare, at least not based on the CPS. This is a tragedy.
It’s not like the CPS hasn’t been revised before. A short paper explaining the 2007 revision lists the more significant changes over the years, including-
- Converting from paper to digital questionnaires in 1994.
- Adding child-specific questions in 1995.
- Adding “verification” questions for people who claimed no insurance in 2000.
The 2007 revision explains how the Bureau did not just stop using one data set and switch to another. Instead, it went back two years and revised the numbers for 2005 and 2007 and provided instruments to allow, “advanced users the ability to approximate the correction for 1997 through 2004.” Both sets of numbers were available for these years to maintain the integrity of the trend lines.
If the Bureau were being responsible it would run the old questionnaire alongside the new one for at least three years, so we could measure the effect of ObamaCare independently of the effect of the new survey.
That it has chosen not to do this only heightens the suspicion of a political agenda in play and further degrades the reliability and trustworthiness of anything that comes out of the federal bureaucracy in the Obama era.
“And to do so at the very moment of the biggest revision of health care in American history is completely irresponsible.”
The Obama Administration’s attempts to mask the statistics to not hurt their political agenda. This should squander any sort of credibility they had left.
For health care researchers and journalists, they should be having a field day with this.
They are probably having a field day criticizing it, but it made their jobs much harder.
The CPS numbers were flawed but useful. Now they will be flawed and biased. There is no way to measure the success or failure of ObamaCare.
Because of these actions, I would measure it as a failure.
If it was an actual success, they would not have taken these measures to change everything. ObamaCare is not a successful as they hoped, and now they are trying to hide it.
“If the Bureau were being responsible it would run the old questionnaire alongside the new one for at least three years, so we could measure the effect of ObamaCare independently of the effect of the new survey.”
It won’t as long as it is run by bureaucrats.
Very good post.
Let me ask a question about this new math:
If 100 million Americans were actively working full time when this administration took over, and only 50 million Americans are still working when they are finished, would they be able to say (again using this “new math”) that they “saved or created” 50 million jobs?
They would say they liberated 50 million people to follow their dreams.
Right, because employment is what is holding people back from following their dreams.
I bet now they dream of paying their bills on time..
Can anything be trusted? No.
Greetings,
Having grown up in Washington one learns as a child, if you don’t like the numbers, change the way you count. Uwe can undoubtedly give us countless examples of this phenomenon–left, right and center.
Cheers,
Charlie Bond
Great post. I only hope that future analyses will note the change in survey questions and the inability to make pre and post comparisons. However my faith in that occurring is nominal.
Since the redesign of the CPS has been in the planning stages for over a decade, did President Obama start cooking these books while still in Illinois … or Kenya?
Progress means change.
Don’t you want your national data collection to be as accurate as possible?
Isn’t more accurate information better than consistently less accurate information? Do you fear that the facts on the ground are better than what you want to see?
http://www.census.gov/hhes/www/hlthins/publications/sehsd_wp_2014-16.pdf
I do indeed want a better survey, Al. I have been saying that for decades. But, as I said, the responsible thing to do would be to run the old and new methods side-by-side for a period of time, like three years, so we can see how they stack up.
Certainly wouldn’t want to play cards with members of the Obama Administration — or participate in their shell game. They keep changing the rules — while the game is in progress. Six and a half years of this nonsense…with Granny-to-be Hillary waiting in the wings. The mind boggles, the stomach churns, and, yeah, health care spending just topped $3 trillion. This leading from behind just ain’t working — anywhere in the world — and costing us a bundle. Rumor has it that Obama’s post-presidency book will be called “Hope You Enjoyed the Fulfillment of My Audacious Dreams — Thanks, Dad”
As a licensed agent who is credentialed to write policies on and off the exchange in nearly half the states, I can tell you that the applications for insurance ask if you have been insured for the past 12 months and, if so, the last day that you were covered.
They have this data. Why aren’t they sharing how many of the 8 million were previously uninsured?
I can tell you that of the plans that I wrote, I had one person who was uninsured who signed up and many more who chose not to purchase and to remain uninsured or to become uninsured due to Obamacare premium increases, network changes, pharmacy formulary adjustments, etc.
Excellent post, Greg.
No.
Next question.