Modeling Wonderland: Errors of 26 Percent to Start

ObamaCare implementation is heavily dependent on central planning informed by economic modeling for risk adjustment, pricing, individual consumption behavior, and estimates of actuarial equivalence. People seem to forget that economic models frequently do a lousy job of predicting real outcomes, and that bad things happen when people who make policy confuse model results with reality.

Here’s what economist Jonathan Gruber wrote about one aspect of this problem in 2011:

It is possible that survey estimates of Medicaid enrollment may differ from the administrative totals in that survey respondents tend to under-report Medicaid coverage. This phenomenon is well-documented and occurs in both state and federal surveys of health insurance coverage.

Administrative totals measure the number of people enrolled and receiving benefits from a program. Surveys sample people and ask them if they are receiving benefits. How different are survey estimates from administrative totals? A lot.

In 2011, survey-based estimates for Colorado Medicaid enrollment were used in planning for the Colorado Health Benefits Exchange. They were 26 percent lower than the number of people on the state’s Medicaid rolls. As Colorado budgeted slightly more than a billion dollars in spending for the Medicaid groups that the estimate covered, a 26 percent underestimate equals about $250 million. For FY 2010-11, that is about 4 percent of the state’s general fund spending, an amount equal to roughly 70 percent of all of the state’s corporate income taxes collections.

Health insurers have historically used past claims experience to predict future costs. But in 2011, the Colorado Health Benefits Exchange Board chose to forecast using the Gruber Microsimulation Model (GMSIM). As explained in the January 2012 report delivered to the Board, the GMSIM model does not rely on extrapolations based on past administrative data. Instead, it depends upon extrapolations from survey data developed from a “number of excellent data sources available for Colorado.” Elizabeth Lukanen, Senior Research Fellow at SHADAC, described GMSIM as the model “used by the executive branch during the development of federal legislation.” According to the report, GMSIM has also been used to inform policy in Massachusetts, California, Connecticut, Delaware, Kansas, Michigan, Minnesota, Oregon, Vermont, Wisconsin and Wyoming.

The surveys used to generate the GMSIM forecasts included a 2008-09 survey of 10,000 randomly selected Colorado households, the Current Population Survey, the Medical Expenditure Panel Survey, and projections from the state and the Congressional Budget Office for growth in Medicaid and “demographic and economic information.” “Imputation methodologies” were developed to fill in data collected by the Current Population Survey but not available in the Colorado Household Survey.

While the quality of the GMSIM forecasts will not be known for a few years, its baseline estimates can now be checked against administrative data. For Colorado in 2011, the surveys generated a GMSIM starting point of 465,000 people in enrolled in “public insurance through the Medicaid or SCHIP programs.” The definition of “public insurance was peculiar.” It excluded the elderly and those who “are disabled covered by Medicaid and Medicare,” generally the most expensive Medicaid enrollees on a per capita basis.

Even when all of the elderly and disabled are subtracted, the GMSIM starting point bears little resemblance to actual enrollments. By December of 2011, Colorado’s official data on Medicaid enrollment reported 620,799 people enrolled. Subtracting official subtotals of adults 65 and older and disabled people of all ages and eligibility leaves 513,271 officially enrolled people. Adding the 74,030 people enrolled in the state’s SCHIP program, Child Health Plan Plus (CHP+), results in 587,301 people on “public insurance” as defined in the GMSIM report. A rough check of the model’s accuracy could have been done using Colorado’s March 2010 official enrollment data, which were certainly available at the time the report was written. At 481,584, 2010 enrollment was already higher than the 2011 GMSIM starting point.

The GMSIM forecasts that ObamaCare will raise premiums in the Colorado individual insurance market by 19 percent in 2016. This increase supposedly accounts for the impacts of ObamaCare’s expansion of benefits, its minimum actuarial benefit requirement, its minimum loss ratio, its guaranteed issue requirement, and the 3 to 1 limit on the premiums charged older versus younger people.

Unfortunately, the model also assumes that “premium reductions due to the managed competition efficiencies introduced by the exchange” will moderate the increase. Facing predicted premium increases of at least 19 percent, people in Colorado’s individual insurance market can only hope that the GMSIM managed competition efficiency theories do a better job of predicting premiums than its survey estimates did in predicting Medicaid enrollments.

Print